Bayes’ Rule, Unreliable Diagnostic Testing, And Containing COVID-19
By: Andy Chen
Tags: Health, Statistics, Medicine, Mathematics

The 2020 Novel Coronavirus Outbreak | Thoughts on Probability and Statistics

Bayes’ Rule, Unreliable Diagnostic Testing, And Containing COVID-19

How false-negatives in diagnostic testing are leading to the release of infected people, motivating extreme containment measures. The COVID-19 outbreak, explained with Bayes’ Rule.

Wuhan coronavirus. Novel coronavirus. COVID-19.

We are currently in February 2020. Over the past month, a deadly virus has been spreading throughout China and the world, sending the infected to the ICU and trapping others in their homes. As authorities try to manage this crisis, they face the challenging issue of containment — sending the infected to quarantine, while allowing the non-infected to go free.

The Problem With Epidemics That Plagues The Authorities

Here is the scenario. You have a cough and a fever. There is a chance that you have caught COVID-19 — the virus spreading throughout the world. You don’t know what this chance is, and you don’t want to take chances, so you seek advice from your doctor.

The issue from the authorities perspective is different. You want treatment, but the authorities need to contain the spread of the virus. From their point of view, there are 4 main outcomes of your visit to the doctor.

If you are infected and diagnosed with coronavirus, they will quarantine you for the public benefit. If you are not infected but you are diagnosed with coronavirus, they will wrongly quarantine you, causing you inconvenience. The public will suffer no major harm, but the authorities will have to expend a small amount of resources. If you are infected and not diagnosed with coronavirus, they will wrongly release you, causing you to spread the virus. This puts the public in grave danger of an outbreak. If you are not infected and not diagnosed with coronavirus, they will rightly release you and save some resources.

The two mistakes that the authorities can make is scenario 2 and 3. Scenario 2 is a minor inconvenience (if not done too often), but scenario 3 is the major issue which can cascade into a larger outbreak, even if only done once. If an outbreak occurs, they will have to do contact tracing for possibly hundreds of people, given the contagiousness and lethality of coronavirus. This will be extremely costly for them, so their primary interest is in minimizing the probability of the third scenario.

Thus, the authorities need to make an accurate diagnosis, so that they can avoid releasing the infected and quarantining the non-infected. To achieve this, the authorities first make an initial assessment of all suspected infections, whether they are patients at the medical clinic, or travellers from places with active outbreaks.

Initial Assessment of a Patient

There are clues which will hint at a COVID-19 infection.

Location: there will be different probabilities of infection for people living in different places. People near the epicenter are more likely to be infected. People living near popular tourist destinations are more likely to be infected than people from isolated places like rural Alaska.

Travel history: travellers from places near the epicenter or other outbreak locations are more likely to have the infection (hence, why travellers are screened).

Social contacts: people who have close contact with the infected/those at risk of infection are more likely to be infected themselves.

Symptoms: people who have symptoms characteristic of coronavirus, such as fever, cough, and shortness of breath are more likely to be infected.

Imaging: people who show imaging features of coronavirus on an x-ray or CT scan are more likely to be infected.

Not all of these clues will be immediately available. Authorities at the airport can only screen people for temperature and identify travel history if needed. A doctor will know about the symptoms disclosed by the patient, but it is on the doctor to take the initiative to find and link the clues together. And an asymptomatic case may not have any clues.

You go to the doctor, and the doctor asks you a series of questions. You tell the doctor about your cough and fever. The doctor is suspicious and builds their own belief of whether you are infected or not. Their belief is strong enough to justify a coronavirus test, so they order you to take this test.

The Probability and Statistics Behind Diagnostic Testing

After gathering enough clues, a doctor may suspect coronavirus. To confirm this suspicion, the doctor will order a diagnostic test.

Misconceptions about what the diagnostic test result means

A diagnostic test is performed by collecting samples from your body (eg. mucus in the back of the nose) and looking for presence of the virus in those samples. It seems simple enough, and people have a lot of faith in science. This may lead people into making this first mistake.

Incorrect interpretation: a positive result means a patient has novel coronavirus, while a negative result means that a patient does not.

This is not true, because the test is not always reliable. There are many reasons why a test may give a misleading result:

A patient in the very early stages of an infection may not excrete a detectable amount of virus.

The virus itself may only exist deeper inside the body, hence being inaccessible by a swab test.

There may have been accidental contamination of the sample.

In general, the people who make the testing kits will specify the reliability of the test.

Suppose that a company now markets their test as “90% accurate”. This can lead to another common mistake.

Incorrect interpretation: for a 90% accurate test, a positive result means 90% chance of being infected, and a negative result means 90% chance of not being infected.

This interpretation is also not true, but is actually surprisingly common in the medical community— the great psychologist Gerd Gigerenzer shows how doctors misinterpret the results of mammogram results article.

Bayesian probability explains what the diagnostic test really means

The correct way to evaluate a diagnostic test requires thinking in terms of Bayesian probability. To put it simply, Bayesian probability involves having a prior probability and then using new information to update it. In terms of diagnostic testing, the prior probability is the doctor’s belief about whether the patient is infected or not. The test result is used as information, and this changes the doctor’s belief.

We can formulate the reliability question in terms of math equations.

Prior probability

The initial probability of infection, which is based on the doctor’s judgement.

Posterior probability: how reliable is a positive result?

The probability of infection is conditioned on a positive result. The positive result is information.

Posterior probability: how reliable is a negative result?

The probability of (no) infection is conditioned on a negative result. The negative result is information.

What we need to do now is to connect the doctor’s initial belief with the final belief. To do this, we use Bayes’ rule, which can easily be derived using basic facts about conditional probabilities.

The denominator is expressed like this.

We now have two new unknown probabilities.

The probability of a positive result given that a patient is infected. This number should be high — infected patients should be getting positive test results. The probability of a positive result given that a patient is not infected . This number should be low — non-infected patients should be getting negative test results.

These two probabilities are actually measures of a test’s reliability, and they can be expressed in terms of two quantities: sensitivity and specificity.

Sensitivity

Probability of a positive result given infection.

The test is “sensitive” to the presence of coronavirus. If the coronavirus is present, the test will detect it.

Ideally, close to 100%.

Specificity

Probability of a negative result given no infection.

The test is “specific” to coronavirus. If there is no coronavirus infection, the test will not detect anything, and returns negative.

Ideally, close to 100%.

I should remind you here that I can express probabilities in terms of complements. Infection and no infection are mutually exclusive. Similarly, a positive and a negative result are also mutually exclusive. This means that our calculations will use these two equations, where A and B are events such as “the patient is infected”, or “the test result is positive”.

The probability of A not occurring is one minus the probability of A occurring.

Given that B has occurred, the probability of A not occurring is still one minus the probability of A occurring.

The probability of infection given a test result

We now have all the tools we need to interpret a test result

Prior probability — initial belief

Posterior probability — final belief

Bayes’ rule — connects initial and final belief

Sensitivity and specificity — allows us to do computations using Bayes’ rule

Combining these, we get