Bayesian Statistics in ML
By: Projjal Gop
Tags: Bayesian Statistics, Statistics, Machine Learning, Predictions

How these odds change

To weigh the new information, and decide how the odds change when it becomes available, we need to consider how likely we would be to encounter this information in alternative situations. Let’s take as an example, the odds that it will rain later today. Imagine getting up in the morning in Finland. The chances of rain are 206 in 365 (including rain, snow, and hail. Brrr). The number of days without rain is therefore 159. This converts to prior odds of 206:159 for rain, so the cards are stacked against you already before you open your eyes.

However, after opening your eyes and taking a look outside, you notice it’s cloudy. Suppose the chances of having a cloudy morning on a rainy day are 9 out of 10 — that means that only one out of 10 rainy days start with blue skies. But sometimes there are also clouds without rain: the chances of having clouds on a rainless day are 1 in 10. Now how much higher are the chances of clouds on a rainy day compared to a rainless day? Think about this carefully as it will be important to be able to comprehend the question and obtain the answer in what follows.

The answer is that the chances of clouds are nine times higher on a rainy day than on a rainless day: on a rainy day, the chances are 9 out of 10, whereas on a rainless day the chances of clouds are 1 out of 10, and that makes nine times higher.

Note that even though the two probabilities 9/10 and 1/10 sum up to 9/10 + 1/10 = 1, this is by no means always the case. In some other towns, the mornings of rainy days could be cloudy eight times out of ten. This, however, would not mean that the rainless days are cloudy two times out of ten. You’ll have to be careful to get the calculations right. (But never mind if you make a mistake or two — don’t give up! The Bayes rule is a fundamental thinking tool for every one of us.)

Likelihood ratio

The above ratio (nine times higher chance of clouds on a rainy day than on a rainless day) is called the likelihood ratio. More generally, the likelihood ratio is the probability of the observation in case the event of interest (in the above, rain), divided by the probability of the observation in case of no event (in the above, no rain). Please read the previous sentence a few times. It may look a little intimidating, but it´s not impossible to digest if you just focus carefully. We will walk you through the steps in detail, just don´t lose your nerve. We’re almost there.

So in our above example of predicting rainy weather, we concluded that on a cloudy morning, we have: likelihood ratio = (9/10) / (1/10) = 9

Formula to calculate posterior odds

The mighty Bayes rule for converting prior odds into posterior odds is — as follows: posterior odds = likelihood ratio × prior odds

Applications

Bayesian models especially naive-bayesian models find their applications in a variety of scenarios like the effectiveness of a testing procedure, classification problems(given the nature of the problem is known), and medical screenings.

That’s it. I hope you like it.

A note from AI In Plain English

We have launched two new publications! Show some love for our new publications by following them: UX in Plain English, Python in Plain English — thank you and keep learning!

We are also always interested in helping to promote quality content. If you have an article that you would like to submit to any of our publications, send us an email at submissions@plainenglish.io with your Medium username and we will get you added as a writer. Also let us know which publication/s you want to be added to.