A short note on Bayes Theorem
By: Balakumaran Panneerselvam
Tags: Decision Making, Mental Models

Let’s say you recently had a COVID test done (I sincerely hope you did not have to). And the test came positive. Let’s say the accuracy of the test is 99%, how sure are you that you indeed have COVID infection? Even with the 99% accuracy, there is a smaller chance that you have the virus.

In this post, let’s have a look at why that would be so.

Let’s take a step back and look at probabilities. Starting with a simple example. When you toss a fair coin, the probability of getting heads is 50% and tails is 50%. It does not mean that you will get 50% head and 50% tail. The outcome will finally be either heads or tails it is just that there is a possibility of getting the outcome is 50%. This is an important distinction.

Another way to look at it is, if I toss the same coin a large number of times, I would get heads 50% of the time and tails 50% of the time.

Another example would be to think of a bet you make on a cricket match. You predict that there is a 30% chance of India winning the match and 70% that India loses the match. You bet 50K on India winning the match. Mathematics tells us the expected payoff of this bet is (30%*50K) + (70%*0) = 15K.

But there is no outcome where you earn 15K. You earn either 50K or 0. The probability of 30% win means that if you were to repeat the bet over independent events a large number of times, you would win 30% of the time and the expected average payout is 15K.

I find the following definition of probability more relevant — probability measures the degree of rational belief to which proposition is entitled in the light of the given evidence

Bayesian Theorem is a way of updating your belief (in turn your probability) with new information. This tells you that Bayesian Theorem deals with base information (Prior Information) and New Information (Posterior Information).

Enough definition, let’s jump into our problem statement — What is the real chance that you have the virus?

Let’s take a sample test with the following characteristics:

On a patient with covid, the test is accurate up to 99% of the time. Meaning, when 100 covid positive patients take the test, 99% of them have a positive result and 1 person has a negative result. We can write this as True positive (TP) = 99%, False Negative (FN) = 1%

On a patient without covid, it is accurate up to 95% of the time. Meaning, when 100 covid negative patients take the test, 95 of them have a negative result and 5 people have a positive result. We can write this as True Negative (TN) = 95% and False Positive (FP) = 5%

Let’s take a sample set of 1000 people taking the test, and you are one of them. Of this 1000, let’s say 10% of the population (100 people) have covid. The results of the test could be expected as the following.

Of the 100 patients with covid, 99 test positive and 1 tests negative.

TP = 99

FN = 1

Of the rest 900 patients without covid, 855 test negative and 45 test positive

TN = 855

FP = 45

So the total positive test results are TP + FP = 99 + 45 = 144. And you are one of them. So, the chances of you having the virus = You being one of the true positive patients among all the patients who tested positive = 99/144 = 68.75%

So even with a test that is accurate up to 99%, the real chances of the test being accurate is only 68.75%! Breathe easy :)

Generalising this,

Prob that a person tested positive has covid = TP/(TP+FP)

If you want to understand the reason behind this, the change is influenced by 2 variables.

What % of the total population being tested is actually infected (In our case, 10%) (What does our base population look like?)

What % of patients without covid have covid positive results (FP). (In our case, 5%)

The above logic is a working of Bayes Theorem. In simple terms, Bayes Theorem takes the behavior of underlying population (Prior Data), adds the new information (FP and TP) and provides the framework to understand the change in probability.

Zooming out, Bayes Theorem provides a mental framework to understand how, rather than looking at new information in isolation, we can improve our thinking by adding new information to the prior knowledge we possess. It is a useful mental model in decision making.

There are fantastic materials on Bayes theorem that are available in the web, I am sharing some useful ones here for additional reading.

https://www.youtube.com/watch?v=HZGCoVF3YvM

https://www.youtube.com/watch?v=R13BD8qKeTg

Book: How not to be Wrong by Jordan Ellenberg