Think Like Bayes
By: Phelix Juma
Tags: Decision Making, Bayesian Inference, Logical Thinking, Bayes Theorem, Critical Thinking

Sheldon Cooper, a Bayesian Thinker, estimates when he is likely to die

“I have addressed this several times. This is not the first time this is coming up. I’ve spoken on it, I’ve addressed it. I’ve said where the rights and wrongs were. I’ve said who I am now vs who I was then…” Kevin Hart was steadfast in his video after quitting from hosting the Oscars. The bone of contention? The Academy of Arts and Sciences wanted him to apologize for a decade old tweets that angered the gay community but Kevin insisted that he had apologized multiple times before and to do so again would be to feed the internet trolls. In his last Netflix Special, “Zero Fucks Given”, Kevin goes ahead to show how the entire world has become “snitches” and how that has affected the industry. In my best comedy Netflix special of all time, “Sticks and Stones”, Dave Chappelle does two impressions: one is of America’s founding fathers while the second one he says is of the audience. In his impression of the audience, Dave laments in his usual genius how the audience has become the feds, they will be laughing at the jokes but at the same time, keeping everything, you say and use them against you even 20 years to come. “Guys, I’m almost 40 years old. If you don’t believe that people change, grow, evolve as people get older, I don’t know what to tell you. If you want to hold people in a position where they always have to justify or explain their past, then do you. I’m the wrong guy.” Kevin posted on his Instagram.

More often than not, we are inclined to hold people to their opinions of old and don’t want to appreciate the fact that these opinions change with time. We brand the people who change their opinions and ideologies with time: “people with no stand”, “traitors”, “disloyal people” and all manner of derogatory phrases but isn’t this what Thomas Bayes advocated against? I have seen Kenyans tell Martha Karua that after supporting Kibaki’s ill-fated swearing in in 2007, she has no right to talk about good and bad 13 years later; we tell Ruto that because he opposed the 2010 constitution, 10 years later, he is being a hypocrite for opposing its amendment. We are basically a bunch of humans who hate progress of the mind; who want people to remain the same way they were decades ago. How then can our society make progress if we can’t allow its people to grow in mind?

What does it mean to think like Bayes? In his book, “A Field Guide to Lies: Critical Thinking in the Information Age”, Daniel J. Levitin says that “Critical thinking is an active and ongoing process. It requires that we all think like Bayesians, updating our knowledge as new information comes in.”

Bayes Theorem

Bayes Theorem tells us that the probability of an event A given B is dependent on the probability of A, the likelihood of B and the probability of B. In Bayesian language, we can say that A is some proposition about the world (a prior belief we hold about certain things in the world) while B is some data or evidence that may or may not influence our beliefs, A. As we start, we have prior beliefs, biases as I called them in my last article, existing “knowledge” or idea of what something should be but as time passes by, we get evidence that either reinforces or disputes the ideas we hold. Bayes theorem thus tells us how we should update our prior beliefs based on the new evidence; the new belief, in Bayesian world, is called posterior knowledge. We don’t stop there, as more data comes in, the new beliefs then form the prior beliefs that we need to update with the new evidence that we get. Basically, every moment is a learning moment and every learning moment would make us to update our existing ideologies and make us become better and different people. If we do this over a span of a decade, the person you knew in 2010 isn’t the same person in 2020 and you cannot hold them to the same beliefs they had in 2010.

Every moment is a learning moment and every learning moment should make us update our existing ideologies and make us become different but better people

In 2010, the Christian, black cultured and naïve teenage me was most likely homophobic but the 2020 me is the exact opposite who understands that just the same way blacks fought for and are still fighting for equality and not wanting to be treated differently because of their skin color, so are the gays fighting for equality and don’t want to be treated differently because of their sexuality. The change hasn’t been abrupt but a miniature change over time of reading about people’s diverse sexuality and what exactly it means; looking at people like Ellen DeGeneres, Jim Parsons, Neil Patrick Harris, Wentworth Miller and they are the best at what they do and you can’t hate them for who they are; you accept them just like everyone else.

Action of evidence on prior beliefs to give rise to our posterior beliefs

This is the same reason why laws, constitutions and such like things are never cast on the stone tablet. I may even go ahead to suggest that the ten commandments need some amendments like allowing people to covet other people’s property and wives/husbands; that seriously doesn’t have to be a sin in the current world. Things like resting on the sabbath is just not possible for some types of service providers eg those working in NOC (Network Operation Center); jobs that never existed when the commandments were made. It’s for the same reason that those who expect the Kenyan BBI to be perfect are as wrong as those who wanted the 2010 constitution to be perfect. Bayes tells us that we must start from somewhere, a prior distribution, and then each day forward is a learning moment. As we get more data and evidence on the performance of the new laws, we get to update our previous beliefs and where need be, do the necessary amendments of the law.

“Under Bayes’ theorem, no theory is perfect. Rather, it is a work in progress, always subject to further refinement and testing.” Nate Silver

Thinking like Bayes cuts across every part of our social, political and professional lives. I was doing a simulation of a marketing campaign on social media in which my mission was to understand the probability of an ad getting a click from the next impression. In this simulation, the past 10 impressions had yielded 7 clicks and so one would be tempted to think that the click through rate is 70% but that’s usually far from the truth.

Likelihood of the probability of a click in the next impression with MLE at 0.7

We have to start from somewhere and that somewhere is the performance of all previous campaigns done. The previous campaigns, as per this simulation, had a click through rate of less than 20%. As the marketing manager starts the new campaign, his belief would be that, because all previous campaigns have had a click through rate of below 20%, the new one would also be around the same value. Once the campaign is posted on Facebook, the manager starts receiving new data on the click through rates; this is the new evidence that the manager needs to use to update his previous belief/estimate of the ad performance. As it turns out, from the simulation, after 10 impressions with 7 click through rates, the manager’s new estimate for the ad performance is a click through rate of 27% and he would continue to do estimates as new data comes in till the end of the campaign. This is the same strategy I employed during the elections in the US in which my prior belief, set at the beginning of the voting process, was that Trump was going to win (I don’t trust opinion polls; article for another day) but as I saw the new data streaming in, half way in the process, I was already sure that Trump was going to lose. That’s how to think like Bayes.

Most plausible probability for a click in the next impression is 27% after we applied evidence to our prior belief

Perhaps the area where Bayesian thinking shines the most is in decision making. A decision may be defined as the process of choosing an action/solution to a problem from a set of feasible alternatives. The biggest question usually is: what action/solution would minimize my expected losses or maximize my profits? A decision-making situation is controlled by three main elements: the alternatives/choices available, states of nature you have no control over and the pay offs ie the net benefit for the decision and how one decides is dependent on the environment in which you are making the decision: is it an environment of certainty, uncertainty or risk? Imagine you’re in a situation where as a doctor, you have to tell a patient that they have x days left to live. What’s the implication of estimating 5 years and the person actually has only 1 year vis a vis estimating 1 year and the person actually has 5 years? Which wrong is worse than the other? Or a second scenario where you either diagnose a patient as HIV positive or negative. If you diagnose negative and yet they are positive, it means they won’t have medication and will die. On the other hand, diagnosing positive while the person is negative means they’ll have side effects from the wrong medication. The side effects are better than death so the second decision would be better wrong. What if you wanted to know when you’re most likely to die? Well, as Sheldon Cooper (played by Jim Parsons in Bing Bang Theory) tries to show us, it seems Bayes has an answer to that question too !

There are so many examples about situations of decision making in which Bayes bails us out. However, the main purpose of this writing is to show us how to think like Bayes; how to be a true Bayesian. To be a true Bayesian is to allow yourself to grow in mind and ideologies based on new evidence and to not allow yourself to be tied down to your ideologies of old. To be a true Bayesian is also to appreciate the Bayesian thinking process; to appreciate that people change, people grow and the same person that hated something a week ago can actually like it today and that’s absolutely ok. A true Bayesian understands that what they know today is not an absolute truth and always learns and seeks for new evidence that either reinforces their beliefs or makes them discard them altogether for better and more sensical beliefs. A true Bayesian knows that the difference between the who I am now vs the who I was then is the new data/evidence I have learned from during that period. A true Bayesian knows that the Kevin Hart of 2008 is not the same Kevin Hart of 2020 and so is everybody else.