Conditional Probability and Bayes' Theorem Examples.
Tags: datascience
Published: 2020-03-23 09:06:31


Last two weeks I was reviewing statistics fundamentals and had to solve few problems using Bayes' Theorem. Here, I will describe a few techniques I found effective in solving common examples using conditional probability. 

When solving these type of problems, I try to solve it ‘intuitively’, if problem is too complicated, then I try to visualize it using probability tree diagram and applying Bayes formula. Finally, if I want to verify my answer, there is an option to do a simulation in Python. 

### **1.Intuitive approach.**

So, what is conditional probability? It is a probability of some event ‘A’ given that another event ‘B’ happen. Basically, event ‘B’ gives us some extra information. Let's see the following examples for better understanding.

####Example 1A. 
A couple has two children. What is the probability that they have two boys?    
There are 4 possible combinations for two children: GG, BB, GB, BG. (Where G=Girl, B=Boy). Probability of two boys is P(BB) = 1/4. This is an example of the probability calculation without conditions (or extra information given).

####Example 1B. 
A couple has two children, one of which is a boy. What is the probability that they have two boys?
Here we have extra information that one of the children is a boy, which narrows possible combinations from 4 to 3: BB, GB, BG. 
Probability that two boys given one is a boy P(BB| one is B) = 1/3. This is an example of conditional probability calculation.

####Example 1C. 
A couple has two children, the older of which is a boy. What is the probability that they have two boys?
Here we have extra information that one of the children is a boy and he is older one, which narrows possible combinations from 4 to 2: BB, BG. 
Probability that two boys given older is a boy P(BB| older is B) = 1/2. This is another example of conditional probability calculation.

####Example 1D.
The Monty Hall problem is a famous little puzzle from a game show. It goes like this: you are presented with 3 doors. Behind two are goats and behind the third is a car. You are asked to select a door; if you select the door with the car, you win! After selecting, the host then opens one of the remaining two doors, revealing a goat. The host then asks if you would like to switch doors or stick with your original choice. What would you do?
Solution:
There are at least few ways to solve this problem, here I chose the simplest approach.
My initial probability that my choice is correct P(my choice is correct) = 1/3, and probability of that my choice is not correct P(not my choice is correct) = 2/3.  But when the host reveals the door with one of the goats,  I should switch because probability of ‘not my choice is correct’ is 2/3 and now represented by one door only. And when I switch I will get 'new' probability = 2/3.

### **2. Construct probability tree diagram.**

For this method we need to use the Bayes' Theorem Formula:

P(A|B) = P(B|A) * P(A) / P(B)  

where
P(A|B) - probability of event A, given event B happen;
P(B|A) - probability of event B, given event A happen;
P(A) - probability of event A;
P(B) - probability of event B; 
and P(B) = P(B|A) * P(A) + P(B|not A) * P(not A).

![](https://dev-to-uploads.s3.amazonaws.com/i/knsewm74a5gqtpvu736a.png)

The probability tree diagram could be handy tool when we need to calculate conditional probability, e.g. P(A|B).  Probabilities shown in red color are in the numerator in Bayes formula P(B|A) * P(A), denominator P(B) includes probabilities shown in the red and green color P(B|A) * P(A) + P(B|not A) * P(not A).
Let's see that on the examples.

####Example 2A. 
A diagnostic test has a probability 0.95 of giving a positive result when applied to a person suffering from a certain disease, and a probability 0.10 of giving a (false) positive when applied to a non-sufferer. It is estimated that 0.5% of the population are sufferers. Suppose that the test is now administered to a person about whom we have no relevant information relating to the disease (apart from the fact that he/she comes from this population). 
Calculate that, given a positive result, the person is a sufferer. 
[Link to the source.](http://wwwf.imperial.ac.uk/~ayoung/m2s1/WorkedExamples1.pdf)

Solution:
a) construct probability tree

![](https://dev-to-uploads.s3.amazonaws.com/i/f1uefkurafl4c9z640xg.png)

b) We need to find P(Disease|Positive Test).

P(Disease|Positive Test) = P(Positive Test|Disease) * P(Disease) / P(Positive Test).

Numerator is shown in red color: 
P(Positive Test|Disease) * P(Disease) = 0.95*0.005 = 0.00475 

Denominator is sum of two branches, shown in red and green:
P(Positive Test) = P(Positive Test|Disease) * P(Disease) + P(Positive Test|no Disease) * P(no Disease) = 0.95*0.005 + 0.1*0.995 = 0.10425

Answer is P(Disease|Positive Test) = 0.00475 / 0.10425 = 0.0455


####Example 2B.
An aircraft emergency locator transmitter (ELT) is a device designed to transmit a signal in the case of a crash. The Altigauge Manufacturing Company makes 80% of the ELTs, the Bryant Company makes 15% of them, and the Chartair Company makes the other 5%. The ELTs made by Altigauge have a 4% rate of defects, the Bryant ELTs have a 6% rate of defects, and the Chartair ELTs have a 9% rate of defects (which helps to explain why Chartair has the lowest market share). 
If a randomly selected ELT is then tested and is found to be defective, find the probability that it was made by the Altigauge Manufacturing Company. 
[Link to the source.](http://faculty.washington.edu/tamre/BayesTheorem.pdf)


Solution
a) construct probability tree

![](https://dev-to-uploads.s3.amazonaws.com/i/ub0awrpmxijk7ssxy2zg.png)

b) We need to find P(Altigauge|Defective).
P(Altigauge|Defective) = P(Defective|Altigauge) * P(Altigauge) / P(Defective)

Numerator is shown in red color: 
P(Defective|Altigauge) * P(Altigauge) = 0.04*0.8 = 0.032

Denominator is sum of three branches, shown in red, green, and purple color:
P(Defective) = 0.04*0.8 + 0.06*0.15 + 0.09*0.05 = 0.0455

Answer is P(Altigauge|Defective) = 0.032 / 0.0455 = 0.7032


### **3. Simulation in Python.**

Here is the simulation of the Monty Hall problem (Example 1D).
```python
import numpy as np
count_switch = 0 #counter for win when switch
count_stick = 0  #counter for win when stick
for i in range(10000):
    car = np.random.choice([1,2,3])    #assign door 1,2, or 3 to car randomly
    player = np.random.choice([1,2,3]) #assign door 1,2, or 3 to player selection randomly
    if car == player:                  
        # If Initial guess is correct and we stick, increase win numbers for 'stick'
        count_stick += 1 
    else:                              
        # If Initial guess is incorrect and we switch, increase win number for 'switch'
        count_switch += 1 
    
P_switch = count_switch/(count_switch+count_stick)
P_stick = count_stick/(count_switch+count_stick)

print('Win number when SWITCH:', count_switch)
print('Win probility when SWITCH:', P_switch)
print('Win number when STICK:', count_stick)
print('Win probility when STICK:', P_stick)
```
Win number when SWITCH: 6641
Win probility when SWITCH: 0.6641
Win number when STICK: 3359
Win probility when STICK: 0.3359

As we can see for 10000 experiments probability of winning when switching is close to 2/3.