Probability of the Challenger Disaster
By: Will Arliss
Tags: Probability, Correlation, Logistic Regression, Challenger, Statistics

By NASA — NASA Human Space Flight Gallery (image link), Public Domain, https://commons.wikimedia.org/w/index.php?curid=181762

Probability of the Challenger Disaster

Introduction

I recently watched the 2020 Netflix docuseries entitled Challenger: The Final Flight, which tells the fascinating yet heartbreaking story of the tragedy of NASA’s Challenger space shuttle. After watching the series, I was inspired to explore the simple statistical modeling that describes the event. For those who have yet to watch the documentary or who are unfamiliar with Challenger’s story, I’ll start with a brief overview.

The 1980s marked a very exciting time for NASA and space exploration. NASA was experiencing success after success in their relatively new Space Shuttle program. Astronauts were being sent to and from orbit on the same vessel, edging NASA closer and closer to their goal of commercial space flight. In 1985, the decision was made to send the first civilian into space.

This decision brought a huge amount of public attention to the STS-51-L mission, which would be Challenger’s 25th trip to space and school teacher Christa McAuliffe’s 1st. On the afternoon of January 28th, 1986 students around America tuned in to watch McAuliffe and six other astronauts launch from Cape Canaveral, Florida. 73 seconds into the flight, the shuttle experienced a critical failure and broke apart in mid air, resulting in the deaths of all seven crewmembers: Christa McAuliffe, Dick Scobee, Judy Resnik, Ellison Onizuka, Ronald McNair, Gregory Jarvis, and Michael Smith.

After an investigation into the incident, it was discovered that the failure was caused by an O-ring in the solid rocket booster. Additionally, it was revealed that such an incident was foreseeable. This blog post will discuss the statistics that could have predicted the critical failure of an O-ring on that day.

O-Ring Failure History

The investigation of the Challenger disaster was carried out by a presidential commission with members running the gambit from politician to general to astronaut to physicist. One member, Nobel laureate Dr. Richard Feynman, demonstrated with a small experiment that the rubber O-rings, used to seal the joints of the fuel tanks for the solid rocket boosters, behave differently under different temperature conditions.

When cooled to a near-freezing temperature, an O-ring will fail to maintain its resilience. In other words, the rubber ring used for sealing the joints of the fuel tanks is more subject to leakage in colder temperatures than in warm ones. This experiment was significant to the commission because the outside temperature in Cape Canaveral on the day of Challenger’s launch was a record low at 36 degrees Fahrenheit — roughly 20 degrees colder than any previous shuttle launch.

The figure below shows a graph from the commission’s report. The graph plots the number of “incidents” per shuttle launch against the calculated temperatures of the O-rings at different launches in the past.

Report of the PRESIDENTIAL COMMISSION on the Space Shuttle Challenger Accident, Chapter IV, [146] Reports, “STS-3 Flight Readiness Review for Levels I, II, III and Contractor.”, Figure 7, https://history.nasa.gov/rogersrep/v1p146.htm

The figure is a little difficult to follow, so I’ve transposed the information below. Here, the “Number of Incidents” has been condensed to simply the occurrence of a failure or not.

There appears to be somewhat of a relationship between failures and the temperature of O-rings here. Visually, the relationship does not appear very strong. Let’s do some analysis.

The Pearson correlation between temperature and failure in these data is -0.5607. This is a fairly strong negative correlation indicating that as temperature decreases, failures increase. We can support this assertion by performing a logistic regression.

Predicting the occurrence of a failure using temperature as the only regressor yields the results in the figure below.

These results indicate, with statistical significance at the 95% confidence level, that a 1 degree increase in temperature decreases the odds of a failure by 0.793. To expand on this inference, a 20 degree decrease in temperature (as was seen with the Challenger explosion) increases the odds of a failure by 15.856.

Below is a plot of the sigmoid curve of this regression. This curve approximates the probability of a failure occurring at different O-ring temperatures.

The probability of failure occurring is extremely high anywhere below 50 degrees Fahrenheit. I was unable to find Challenger’s O-ring temperature on the day of the fatal launch, so the blue X in the upper left corner of the plot instead marks the outside temperature. At this temperature, these data and the associated model give a probability of over 0.99 for a failure occurring.

We can see now that the chances of an O-ring failure occurring on the day of Challenger’s launch were incredibly high. For whatever reason, the NASA and Thiokol engineers were unable to effectively communicate this to those in charge of scheduling the launch. Read up on the myriad factors that allowed the launch to happen in this article.

Conclusion

I concede that the benefit of hindsight makes it much easier to predict an incident like this. After all, when you start with an outcome and then look for the evidence, you’re more likely to succeed than if you start with evidence and look for an outcome. Regardless, the intuition needed for predicting an incident on the day of the Challenger explosion is held by most sophomore Statistics students.

I hope to have made clear the simple methods necessary for predicting the outcome of the Challenger launch. For more insights into the events leading up to and following the disaster, I highly recommend the docuseries mentioned at the top of this post.

References