Bayes' Theorem In Layman Terms
Tags: statistics, python, ai, machinelearning
Published: 2023-08-28 07:55:14

# DEFINITION
_Bayes' Theorem states that the conditional probability of an event, based on the occurrence of another event, is equal to the likelihood of the second event given the first event multiplied by the probability of the first event._

#### Blaa Blaa Blaa - I find definitions to be strange, only after understanding the concept I do understand the definition.

# Let's break it down and understand it one step at a time...

## Marginal Probability - P(A)
If a [random variable is independent](https://www.probabilitycourse.com/chapter3/3_1_4_independent_random_var.php) then the probability of the event is irrespective of the outcomes of other random variables. In simple words, it's like looking at the probability of something occurring without taking into account any other factors.

## Joint Probability - P(A,B)
The probability of 2 or more **simultaneous** events happening together. Eg Probability of watching TV and Eating.

## Conditional Probability - P(A|B)
Probability of one (or more) event given the occurrence of another event. Eg the probability of your father having dessert given that tomorrow he is having a diabetes test is very low. If you notice carefully if there is no diabetes test tomorrow then the probability would have been almost 100%.

## Expressing Joint Probability In Terms of Conditional Probability
{% katex %} P(A,B) = P(A|B) * P(B) {% endkatex %}
Note : **P(A,B) = P(B,A)** (Symmetrical)

## Expressing Conditional Probability In Terms of Joint Probability

{% katex %} P(A|B) = P(A,B) / P(B) {% endkatex %}

Note : **P(A|B) != P(B|A)** (Not Symmetrical)

## Finally our Bayes' Theorem using the above equations

{% katex %} P(A|B) = (P(B|A) * P(A))/P(B) {% endkatex %}

The numerator **P(B|A) * P(A)** is the **joint probability** equation given in **(1)**
P(A|B) ===> **Posterior Probability**
P(B|A) ===> **Likelihood**
P(A)   ===> **Prior Probability**
P(B)   ===> **Evidence**

### To understand the above equation with an example
Question: What is the probability that there is fire given that there is smoke?

{% katex %}P(Fire|Smoke)=P(Smoke|Fire)*P(Fire)/P(Smoke){% endkatex %} 

P(Fire|Smoke) ===> **Posterior Probability**
P(Smoke|Fire) ===> **Likelihood**
P(Fire) ===> **Prior Probability**
P(Smoke) ===> **Evidence**


The probability of fire given that there is smoke is equal to the likelihood multiplied by the probability of fire divided by the probability of smoke. And this is Bayes' theorem to understand its use-case better read further.

## Where is Bayes' Theorem used and why Bayes' Theorem
One very common space where you can find the theorem applied is in the evaluation of **medical diagnostic tests**.


![Cancer Test](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e830eu79wxy42c58mr7v.png)

Let us consider a diagnostic test that determines whether a person has a lesion that is malignant or not.

From observation, it is given that
{% katex %}P(Test=Positive|Malignant=True) = 0.85 {% endkatex %}
The above statement means that the probability of diagnostic test results being **Positive** given that he/she has a malignant tumour is **85%**.

What will be a normal person's understanding of the above probability??? 
If a person takes this diagnostic test and the **result turns out to be Positive** since the above statement shows that for someone with a Malignant tumour, the test detects **85% per cent** correctly there is a good chance that the person assumes that he/she might have a malignant tumour and that's scary, right? 

Now let's look at what Bayes' got to say about it
**P(Malignant=True|Test=Positive)**, this is what we are going to analyse using Bayes' theorem

There are a few assumptions that we have to make

{% katex %}P(Malignant) = 0.0002 {% endkatex %}
{% katex %}P(Test=Positive) = 0.05016 {% endkatex %}

This assumption means that on average only **1** in **5000** will have malignant tumours and the probability of the test is **positive** regardless of whether the person has a **malignant tumour or not is 0.05016**.


{% katex %}P(Malignant=True | Test=Positive) = P(Test=Positive|Malignant=True) * P(Malignant=True) / P(Test=Positive) {% endkatex %}

Plugging the values that we have

{% katex %} P(Malignant=True | Test=Positive) = 0.85 * 0.0002 / 0.05016
{% endkatex %}

### P(Malignant=True | Test=Positive) = 0.003389
Wait what... this is a **terrible diagnostic test** because the the above probability shows that if this diagnostic test for a malignant tumour turns out to be **True** the probability of it being **correct** is only **0.33** per cent.
**_Note_**: This result was obtained on a few assumptions and if those assumptions are verified and updated the result could change.

##CONCLUSION
Bayes' theorem is a significant contribution to the field of statistics and is widely used in machine learning. Bayes' Theorem provides a systematic way to update prior probabilities with new information or evidence. In other words, it helps us adjust our beliefs about the likelihood of an event occurring based on the data we observe.

LinkedIn : https://www.linkedin.com/in/praveenr2998