Anthrax Cleanup Decisions: Statistical Confidence or Confident Response

“C you guarantee this building is safe?” This question is difficult to answer in the wake of a biological attack. It is likewise difficult to decide when people may return to a place where dangerous agents were once dispersed. This decision becomes especially challenging when even minute quantities of the suspected contaminating agent could pose a significant threat (e.g., weaponized anthrax). Following the 2001 US anthrax attacks, the Government Accountability Office (GAO) directed the U.S. Department of Homeland Security (DHS) to develop a defensible strategy for making such decisions following biological incidents. Statistically based sampling and analysis has historically provided the information to guide remediation of contaminated sites. However, classical statistics approaches could require thousands of tests to conclude that the number of anthrax spores is below the level of 0.1 spores per square meter (a concentration which could still pose a significant risk). Supplemental experimental data (e.g., dispersion studies, preliminary sampling) may be difficult to obtain due to financial constraints and could be limited to specific experimental conditions. Moreover, classical statistics expresses results as confidence or tolerance intervals and does not actually state a probability of contamination necessary for risk-based decision making, which poses a challenge of communicating test results in a meaningful way. For example, the confidence interval statement, “with 90% confidence the probability of contamination is less than 5%,” means “if the probability of contamination was really 5% or greater, there is only a 10% chance we would have observed no contamination.” Likewise, the tolerance interval statement “with 90% confidence, 95% of the room has no contamination” is not reassuring. Modern Bayesian statistical approaches (developed starting in the 1950s, building from a tradition going back to Bayes and Laplace in the 1700s-1800s) facilitate inferences about the probability of remaining contamination, effectively overcoming much of the aforementioned challenge. Bayesian methods can smoothly incorporate data from laboratory experiments (e.g., decontaminating different surfaces such as steel and concrete), which is useful when conditions are dynamic. Additionally, when relevant expert knowledge exists, Bayesian statistics can smoothly incorporate it into decision making, in place of further sampling. Finally, Bayesian methods can support direct statements about: probabilities, e.g., “there is a 95% probability that the room is not contaminated”; probability ranges, e.g., “are 95% confident that the probability of the room being clean is between 2% and 4%”; or probability distributions, e.g., “the probability distribution over the number of spores follows a beta distribution with these parameters” or even “there is a 1% chance that there are ten or more spores remaining” (which could be useful in less hazardous situations where some risk of exposure may be tolerable). An important caveat is that the probability statements are still fundamentally an assertion of the expert beliefs based on subjective inputs regarding properties of contaminants and cleanup methods under different conditions tempered by the logical implications of the observed data. They should not be viewed as a way to “launder” opinions into fact, and if experts do not know enough to provide strong judgments, new data will still be needed to gain adequate certainty about treatment success. The other key limitation of the Bayesian approach is that people— including subject matter experts—are known to have a variety of systematic biases in making subjective probability estimates. Scientific research is replete with examples of overconfidence, where experts were slow to update their beliefs in the face of new