Conditional Probability with Python: Concepts, Tables & Code
Tags: python, datascience, machinelearning
Published: 2020-11-23 15:12:02

## Overview


This post is chapter 6 in continuation of my coverage of [Data Science from Scratch by Joel Grus](https://joelgrus.com/2019/05/13/data-science-from-scratch-second-edition/). We will work our way towards understanding conditional probability by understanding preceding concepts like marginal and joint probabilities. 

At the end, we'll tie all concepts together through code. For those inclined, you can jump to the code towards the bottom of this post. 


## Challenge

The first challenge in this section is distinguishing between **two** conditional probability statements. 

Here's the setup. We have a family with two (unknown) children with two assumptions. First, each child is equally likely to be a boy or a girl. Second, the gender of the second child is *independent* of the gender of the first child.

> Outcome 1: What is the probability of the event "both children are girls" (B) conditional on the event "the older child is a girl" (G)?

The probability for statement one is roughly 50% or (1/2).

> Outcome 2: What is the probability of the event "both children are girls" (B) conditional on the event "at least one of the children is a girl" (L)?

The probability for statement two is roughly 33% or (1/3).

But at first glance, they look similar. 

## Marginal and Joint Probabilities 

The book jumps straight to conditional probabilities, but first, we'll have to look at **marginal** and **joint** probabilities. Then we'll create a **joint probabilities table** and **sum** probabilities to help us figure out the differences. We'll then *resume* with **conditional probabilities**. 

Before anything, we need to realize the situation we have is one of **independence**. The gender of one child is **independent** of a second child. 

The intuition for this scenario will be different from a **dependent** situation. For example, if we draw two cards from a deck (without replacement), the probabilities are different. The probability of drawing one King ♠️ is (4/52) and the probability of drawing a second King ♣️ is now (3/51); the probability of the second event (a second King) is *dependent* on the result of the first draw. 

Back to the two unknown children. 

We can say the probability of the first child being either a boy or a girl is 50/50. Moreover, the probability of the second child, which is **independent** of the first, is *also* 50/50. Remember, our first assumption is that *each child is equally likely to be a boy or a girl*.

Let's put these numbers in a table. The (1/2) probabilities shown here are called **marginal** probabilities (note how they're at the margins of the table).

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/mji15tnha84xacdwyhg5.png)

Since we have two gender (much like two sides of a flipped coin), we can intuitively figure out *all* possible outcomes:

1. first child (Boy), second child (Boy)
2. first child (Boy), second child (Girl)
3. first child (Girl), second child (Boy)
4. first child (Girl), second child (Girl)

There are *4 possible outcomes* so the probability of getting any one of the four outcomes is (1/4). We can actually write these probabilities in the middle of the table, the **joint probabilities**:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/nioci6co3ixrik3fykqc.png)

To recap, the probability of the first child being either boy or girl is 50/50, simple enough. The probability of the second child being either boy or girl is also 50/50. When put in a table, this yielded the **marginal probability**. 

Now we want to know the probability of say, 'first child being a boy and second child being a girl'. This is a **joint probability** because is is the probability that the first child take a specific gender (boy) **AND** the second child take a specific gender (girl).

If two event are **independent**, and in this case they are, their **joint probabilities** are the *product* of the probabilities of **each one happening**.

The probability of the first child being a Boy (1/2) **and** second child being a Girl (1/2); The product of each marginal probability is the joint probability (1/2 * 1/2 = 1/4).

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/en8ssu7alddsplwhrbwi.png)

This can be repeated for the other three joint probabilities. 

## Conditional Probability

Now we get into **conditional probability** which is the probability of one event happening (i.e., second child being a Boy or Girl) **given that** or **on conditional that** another event happened (i.e., first child being a Boy).

At this point, it might be a good idea to begin writing probability statements *similar* to how it is expressed in mathematics. 

A joint probability is the product of each individual event happening (assuming they are independent events). For example we might have two individual events:

- P(1st Child = Boy): 1/2
- P(2nd Child = Boy): 1/2

Here is their **joint probability**:
- P(1st Child = Boy, 2nd Child = Boy)
- P(1st Child = Boy) * P(2nd Child = Boy) 
- (1/2 * 1/2 = 1/4)

There is a relationship between **conditional** probabilities and **joint** probabilities. 

Here is their **conditional probability**:
- P(2nd Child = Boy | 1st Child = Boy) 
- P(1st Child = Boy, 2nd Child = Boy) / P(1st Child = Boy)

This works out to: 
- (1/4) / (1/2) = 1/2
or 
- (1/4) * (2/1) = 1/2

In other words, the probability that the second child is a boy, given that the first child is a boy is still 50% (this implies that with respect to **conditional** probability, if the events are **independent** it is not different from a single event). 

Now we're ready to tackle the two outcomes posed at the beginning of this post.

> Outcome 1: What is the probability of the event "both children are girls" (B) conditional on the event "the older child is a girl" (G)?

Let's break it down. First we want the probability of the event that "both children are girls". We'll take the product of two events; the probability that the first child is a girl (1/2) and the probability that the second child is a girl (1/2). So for **both** child to be girls, 1/2 * 1/2 = 1/4

- P(1st Child = Girl, 2nd Child = Girl) = 1/4

Second, we want that to be **given that** the "older child is a girl". 

- P(1st Child = Girl) = 1/2

**Conditional probability**: 
- P(1st Child = Girl, 2nd Child = Girl) / P(1st Child = Girl)
- (1/4) / (1/2) = (1/4) * (2/1) = (2/4) = **1/2** or roughly **50%**

Now let's break down the second outcome: 

> Outcome 2: What is the probability of the event "both children are girls" (B) conditional on the event "at least one of the children is a girl" (L)?

Again, we start with "both children are girls":

- P(1st Child = Girl, 2nd Child = Girl) = 1/4

Then, we have "on condition that at least one of the children is a girl". We'll reference a **joint probability table**. We see that when trying to figure out the probability that "at least one of the children is a girl", we rule out the scenario where **both** children are boys. This is actually the [compliment](https://www.statsandr.com/blog/the-9-concepts-and-formulas-in-probability-that-every-data-scientist-should-know/#complement-of-an-event) to *at least one child is a girl*.  The remaining 3 out of 4 probabilities, fit the condition. 

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/1t162atqsozuce4lajfe.png)

The probability of at least one children being a girl is:
- (1/4) + (1/4) + (1/4) = 3/4

So:
- P(1st Child = Girl, 2nd Child = Girl) / P("at least one child is a girl")
- (1/4) / (3/4) = (1/4) * (4/3) = (4/12) = **1/3** or roughly **33%**

#### Key Take-away

When two events are **independent**, their **joint probability** is the product of each event:
- P(E,F) = P(E) * P(F)

Their **conditional** probability is the **joint probability** divided by the conditional (i.e., P(F)).
- P(E|F) = P(E,F) / P(F)

And so for our two challenge scenarios, we have:

Challenge 1:
- B = probability that both children are girls
- G = probability that the *older* children is a girl

This can be stated as: P(B|G) = P(B,G) / P(G)

Challenge 2:
- B = probability that both children are girls
- L = probability that *at least one* children is a girl

This can be stated as: P(B|L) = P(B,L) / P(L)

#### Python Code

Now that we have an intuition and have worked out the problem on paper, we can use code to express conditional probability:

```python
import enum, random
class Kid(enum.Enum):
    BOY = 0
    GIRL = 1
    
def random_kid() -> Kid:
    return random.choice([Kid.BOY, Kid.GIRL])
    
both_girls = 0
older_girl = 0
either_girl = 0
random.seed(0)
for _ in range(10000):
    younger = random_kid()
    older = random_kid()
    if older == Kid.GIRL:
        older_girl += 1
    if older == Kid.GIRL and younger == Kid.GIRL:
        both_girls += 1
    if older == Kid.GIRL or younger == Kid.GIRL:
        either_girl += 1
        
print("P(both | older):", both_girls / older_girl)   # 0.5007089325501317
print("P(both | either):", both_girls / either_girl) # 0.3311897106109325
```
We can see that code confirms our intuition by looking at each of the **joint probabilities**

```python
either_girl #7,464 / 10,000 ~ roughly 75% or 3/4 probability that there is at least one girl
both_girls  #2,472 / 10,000 ~ roughly 25% or 1/4 probability that both children are girls
older_girl  #4,937 / 10,000 ~ roughly 50% or 1/2 probability that the first child is a girl
```

#### Code Breakdown

Challenge 1:
- P(B|G) = P(B,G) / P(G)
or more explicitly:
- P(both_girls | older_girl) = P(both_girls) / P(older_girl)

Challenge 2:
- P(B|L) = P(B,L) / P(L)
or more explicitly:
- P(both_girls | either_girl) = P(both_girls) / P(either_girl)

**Conditional** probabilities are **conditional** statements in code.

First, we establish a random function that assigns a `random.choice` method to assign gender such that each child (i.e.,`Kid` class instance) *is equally likely to be a boy or a girl*. This is the first assumption of our scenario.

```python
import enum, random

class Kid(enum.Enum):
    BOY = 0
    GIRL = 1
    
def random_kid() -> Kid:
    return random.choice([Kid.BOY, Kid.GIRL])
```
Next we create variables representing **joint distributions**; one variable for both children being girls (`both_girls`), one variable for only the *older child* being a girl (`older_girl`), and one for *at least one* child being a girl (`either_girl`). 

First the probability of any one child being a girl is (1/2), consistent with our assumption, we'd expect:

```python
older_girl  #4,937 / 10,000 ~ roughly 50% or 1/2 probability that the first child is a girl
```
Recall that when we take the **product** of *each* child being a girl (1/2), we can figure out the joint probability of *both* child being a girl (1/4). Thus, we'd expect:

```python
both_girls  #2,472 / 10,000 ~ roughly 25% or 1/4 probability that both children are girls
```

Finally, recall that if we're trying to calculate that *at least one (of two) children is a girl*, we can rule-out the (1/4) probability that both children are boys leaving (1/4 + 1/4 + 1/4 = 3/4) (see table above). Thus, we'd expect:

```python
either_girl #7,464 / 10,000 ~ roughly 75% or 3/4 probability that there is at least one girl
```

To arrive at the numbers we see above, we create 10,000 *simulations* of scenarios where the 1st Child and 2nd Child (see table above) are randomly assigned gender in each of the scenario and conditionally filter through the code to see if certain outcomes are `True`.

```python
random.seed(0)

for _ in range(10000):
    younger = random_kid()
    older = random_kid()
    if older == Kid.GIRL:
        older_girl += 1
    if older == Kid.GIRL and younger == Kid.GIRL:
        both_girls += 1
    if older == Kid.GIRL or younger == Kid.GIRL:
        either_girl += 1
```

This simulation yields the joint probabilities which are then used to find the conditional probabilities of the two outcomes above:

```python
print("P(both | older):", both_girls / older_girl)   # 0.5007089325501317
print("P(both | either):", both_girls / either_girl) # 0.3311897106109325
```
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/9p7u76m65ig11y22v4l5.png)
_____

For more content on data science, machine learning, R, Python, SQL and more, [find me on Twitter](https://twitter.com/paulapivat).