Python vs. Goats: Monty Hall revisited | by Norbert Widmann | Towards Data Science

The photo of playing Pandas notwithstanding, we going to talk about the Monty Hall problem here. This problem is famous in statistics and probably has introduced the majority of statistics courses in university, online or anywhere. What most people including me learn from this problem is that statistical or probabilistic thinking is unfortunately counterintuitive and therefore has to be trained. Or in other words: Just learning the formulas will not get you all that far beyond passing the course.
But now back to the actual problem: You are the lucky person who participates in a game show. The main price is an expensive sports car, probably battery electric nowadays. Between you and driving away in a brand new car are three doors. Behind two of the doors are goats and behind one door the car of your dreams. How can you defeat the goats and get the car?
The rules of the game are: You choose a door. Then Monty, the talk show host, chooses another door, opens it and behind it is invariably a goat. So one of the goats is already out of the way. Now what do you do? Change to the second closed door or stick with the lucky feeling that made you choose your door. My first intuition said it does not matter: three doors, one car makes a one third chance. Monty’s action does not matter, he always chooses a goat. Or does it?
Now mathematics says it does, especially taking into account the work of some Thomas Bayes. I read quite a few explanations of the problem, also on Medium. None of them struck as me as immediately understandable. So why not just try it out? Being out of an unlimited amount of goats and sports cars I chose to simulate using Python with some help from Pandas.
The problem can easily be broken down into the following steps:
We hide a car behind one of the three doors and two goats behind the other two.We choose a door, due to lack of further information at random, just following our lucky feeling.Monty, the friendly quiz show host, choose one of the other two doors, opens it and invariably a goat walks out of it.He offers us to change doors. What do we do?Our door is opened. We either drive into the sunset in a brand new car or have to accomodate a new pet.
We hide a car behind one of the three doors and two goats behind the other two.
We choose a door, due to lack of further information at random, just following our lucky feeling.
Monty, the friendly quiz show host, choose one of the other two doors, opens it and invariably a goat walks out of it.
He offers us to change doors. What do we do?
Our door is opened. We either drive into the sunset in a brand new car or have to accomodate a new pet.
There is only one decision point in step 4. We will aid our Python in this step by defining three scenarios and simulating all three of them:
Scenario A: Always change.
Scenario N: Never change, stick with that lucky feeling.
Scenario S: Sometimes change, due to lack of better understanding of the problem with a 50:50 chance between the two unopened doors.
In which scenario can we defeat the goats?
Clearly, we are talking about probabilities here. So unfortunately neither mathematics nor Python can help the lucky quiz show guest in always getting the sports car. But Python can help us in getting the probabilities and actually believing them. Something where mathematics at least for me always had its difficulties getting me. I will explain a bit about the code here.
The goal was to make the code as readable and understandable as possible and sacrifice efficiency both in memory and performance for it. So you will be able to find much more concise and efficient simulations of the problem. We start by modelling the Monty Hall problem using a Pandas dataframe. Note the number_games is an integer defining the number of games we want to play.
all_games = pd.DataFrame(index=range(0, number_games),                          columns=['Door_A', 'Door_B', 'Door_C',                            'Choice_1', 'Choice_Monty',                             'Choice_A', 'Choice_N', 'Choice_S',                            'Success_A', 'Success_N', 'Success_S'])for game_idx in all_games.index:    # we put goats behind all doors    all_games.loc[game_idx, ["Door_A", "Door_B", "Door_C"]] =                             ["Goat", "Goat", "Goat"]    # oops, we forgot the car    all_games.loc[game_idx, random.choice(["Door_A", "Door_B", "Door_C"])] = "Car"    # the first choice is purely random    all_games.loc[game_idx, "Choice_1"] = random.choice(["Door_A", "Door_B", "Door_C"])
If you take a look at the generated Pandas dataframe you will see that for all the games we hid a car behind one of the Doors A to C and two Goats behind the other two. We already picked a door. The next step is Monty’s choice and this is where the magic happens. His choice is far from random.
for game_idx in all_games.index:    game = all_games.iloc[game_idx]    # Monty will not open the door we chose    monty_choices = {"Door_A", "Door_B", "Door_C"} - {game["Choice_1"]}    monty_choices = list(monty_choices)    # Monty will not open the door with the car    if(game[monty_choices[0]] == "Car"):        monty_choice = monty_choices[1]    elif(game[monty_choices[1]] == "Car"):        monty_choice = monty_choices[0]    else:        monty_choice = random.choice(monty_choices)    # We store Monty's choice    all_games.loc[game_idx, "Choice_Monty"] = monty_choice
After Monty made his choice we apply our three strategies as defined above: A for always change, N for never change and S for sometimes change with a 50:50 probability.
# never changeall_games["Choice_N"] = all_games["Choice_1"]for game_idx in all_games.index:    game = all_games.iloc[game_idx]    # always change    all_games["Choice_A"][game_idx] = ({"Door_A", "Door_B", "Door_C"} - {game["Choice_1"], game["Choice_Monty"]}).pop()    # sometimes change    all_games["Choice_S"][game_idx] = random.choice(list({"Door_A", "Door_B", "Door_C"} - {game["Choice_Monty"]}))
The code should be readable even if you are not a Python aficionado. In the always change case we need the pop() to get the only remaining element from the set. The last step is scoring our scenarios with 1 if we actually got the car and 0 if we are left with a somewhat smelly four legged addition to our household. I will leave that to the reader, alternatively you can get the source code on my GitHub.
Once we have scored the simulation we can simply use describe() on our dataframe to get the results. In one of my runs of 10,000 games I got the folowing values:
Scenario A: 0.6659 probability of getting the car if we always change.
Scenario N: 0.3341 probability if we never change,
Scenario S: 0.4980 probability if we sometimes change.
Looking at the results of our code we can see that the scenario A of always switching doors leaves us with a two thirds probability of getting the sports car. So in the long run 2 out of 3 times we will get the car instead of a useless goat. Why is that so?
Following scenario N to never change we stick with the situation of three doors, no previous information, the chance of one third. But Monty shows us one door with a goat. So he transfers the problem into a two-door 50:50 chance problem of getting the car. This is reflected in the scenario S to sometimes change doors.
But there is a third alternative reflected in scenario A to always change. With our choice, we partition the doors into two sets: Our door with a one third chance of holding the car and the other two doors with a two thirds chance. So our choice adds information by doing exactly this partitioning.
Now Monty looks only at the two door partition with the two thirds chance. After Monty’s choice we can be sure that the unopened door in that partition hides the car if it was there in the first place. So we inherit the two thirds probability by always choosing that door.
Still difficult to understand? Well, yes, I agree. But just look at the numbers the simulation gave us. Simulating in Python helped me in defeating the goats and get a better understanding of the problem. Hopefully, it will do the same for you! Just get the code and try for yourself for example 10,000 times. This will give you a reasonably exact estimate of the probabilities. How exact? I have to go back to my statistics courses for that one.