The Sleeping Beauty Problem
Tags: math, probability, statistics
Published: 2023-07-17 17:20:12

Sleeping Beauty agrees to be the subject of an experiment and is informed of the procedure:

* On Sunday night, she will be put to sleep.
* Then, a fair coin will be flipped.
* If it comes up Heads, she will be awakened on Monday and put back to sleep.
* If it comes up Tails, she will also be awakened on Monday and put back to sleep, but then she'll be awakened on Tuesday as well and put back to sleep.
* Each time that she is put back to sleep she will forget that she was ever awakened. 
* Any time she's awake, she will be told no information but she'll be asked one question: **What do you believe is the probability that the coin came up heads?**


![Experiment](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2kjuxhi134mnyly8gkr4.png)
###### Image credit: [Veritasium](https://www.youtube.com/@veritasium) 

How should she answer?

You might be tempted to say {% katex inline %}50\%{% endkatex %} because that's the probability of a fair coin coming up Heads, but you would be mistaken. The correct answer is {% katex inline %}1/3{% endkatex %}. To understand why, we need to realize that the question that is *really* being asked is this: What is the probability that the coin came up Heads given that you've been awakened?

This is a case where we need to incorporate observed data into our prior odds to come up with new odds that account for the new data. To do that, we need to calculate the probability of an event happening given that another event has already happened. We can use Bayes' Theorem for this.

> :information_source: **Bayes' Theorem**
> {% katex %}P(A|B)={P(B|A)P(A) \over P(B)}{% endkatex %}

But wait, where did that come from?
> :bulb: It can be derived from the conditional probability that tells us in how many cases do both events {% katex inline %}A{% endkatex %} and {% katex inline %}B{% endkatex %} happen out of the cases where {% katex inline %}B{% endkatex %} happens:
>
> {% katex inline %}P(A|B)={P(A \cap B) \over P(B)}{% endkatex %}
>
> Similarly, {% katex inline %}P(B|A)={P(A \cap B) \over P(A)}{% endkatex %}
>
> which means
>
> {% katex inline %}P(A \cap B)=P(B|A)P(A){% endkatex %}
>
> and substituting in the expression for conditional probability yields Bayes' Theorem:
> {% katex %}P(A|B)={P(B|A)P(A) \over P(B)}{% endkatex %}

Now that we understand where Bayes' Theorem comes from, there's another way to look at it: As a way to understand two competing hypotheses {% katex inline %}H_1{% endkatex %} and {% katex inline %}H_2{% endkatex %}. We start by having some prior belief about the odds of those two. After observing some data, what are the **new** odds of the two hypotheses given the new data? In other words, how do we update our belief based on new evidence?

> :information_source: 
> Let's start by specifying what we mean by *odds*. We mean how large is one probability relative to the other. Let's put it as a ratio. For example, if prior to seeing any data one hypothesis has {% katex inline %}3/4{% endkatex %} probability of being true and the other has {% katex inline %}1/4{% endkatex %}:
> 
> {% katex %}\frac {P(H_1)} {P(H_2)}=\frac {3/4} {1/4}=\frac 3 1{% endkatex %}
> 
> Meaning the odds of {% katex inline %}H_1{% endkatex %} to {% katex inline %}H_2{% endkatex %} are {% katex inline %}3{% endkatex %} to {% katex inline %}1{% endkatex %}.

So how do we incorporate the new data {% katex inline %}D{% endkatex %} into our belief? 

> :bulb: By calculating the new probability of each hypothesis given the new data.
> 
> {% katex %}\frac {P(H_1|D)} {P(H_2|D)}=\frac {\frac {P(H_1 \cap D)} {P(D)}} {\frac {P(H_2 \cap D)} {P(D)}}=\frac {P(H_1 \cap D)} {P(H_2 \cap D)}{% endkatex %}
> 
> Now we'll use a mathematical trick of multiplying and dividing **by the same number** to get the same value but expressed in terms of the **probability of seeing the observed data given each hypothesis**.
> 
> {% katex %}=\frac {P(H_1 \cap D)} {P(H_2 \cap D)} \cdot \frac {P(H_2)} {P(H_1)} \cdot \frac {P(H_1)} {P(H_2)}{% endkatex %}
> 
> {% katex %}=\frac {\frac {P(H_1 \cap D)} {P(H_1)}} {\frac {P(H_2 \cap D)} {P(H_2)}} \cdot \frac {P(H_1)} {P(H_2)}{% endkatex %}
> 
> So,
> {% katex %}\frac {P(H_1|D)} {P(H_2|D)}=\frac {P(D|H_1)} {P(D|H_2)} \cdot \frac {P(H_1)} {P(H_2)}{% endkatex %}
> 
> And so we have that as a ratio, the posterior odds equal the prior odds times the probabilities of generating the new data.

Now let's apply this to our Sleeping Beauty problem. Let
{% katex inline %}H_1={% endkatex %}*The coin came up Heads*
{% katex inline %}H_2={% endkatex %}*The coin came up Tails*
{% katex inline %}D={% endkatex %}*You've been woken up*

{% katex %}\frac {P(H_1|D)} {P(H_2|D)}=\frac {1/3} {2/3} \cdot \frac {1/2} {1/2}=3/6=1/2{% endkatex %}

Meaning {% katex inline %}H_2{% endkatex %} is twice as likely as {% katex inline %}H_1{% endkatex %} given the new data. If you want to convert that to probabilities: 

{% katex %}P(H_1)=\frac 1 {1+2}=1/3{% endkatex %}

{% katex %}P(H_2)=\frac 2 {1+2}=2/3{% endkatex %}

Therefore Sleeping Beauty should answer that the coin came up Tails as the probabilities are {% katex inline %}1/3{% endkatex %} for Heads and {% katex inline %}2/3{% endkatex %} for Tails.

---

We can give another explanation that is more informal but more intuitive. Imagine she's still woken up once if it's Heads, but on {% katex inline %}100{% endkatex %} consecutive days if it's Tails. She's awake. She has no idea which of those {% katex inline %}101{% endkatex %} scenarios she's in, but effectively she's being asked if she thinks she's in the {% katex inline %}1{% endkatex %} scenario where it was Heads or in one of the {% katex inline %}100{% endkatex %} scenarios where it was Tails. The answer should be clear.