On the difference between inference and prediction
By: Tomasz Wesołowski
Tags: Data Analysis, Statistics, Data, Data Science, Predictions

Inference

The inference approach is by far the most common one. The question it helps to unravel is: Who’s going to win the election? Of course, we cannot ask every single voter about their preference — it would cost us millions of dollars and an infeasible amount of time (and thanks God for this — otherwise, we would need to watch election debates every weekend!). That’s why pollsters usually select a small, representative group of people and they try to infer the opinion of the whole nation based on the small sample selected.

This is basically what inference is all about. In general, the goal of inference is to reach a conclusion about the property of the population of interest. Usually, this property is not directly observable, either because we do not have the right tools or because the costs outweigh the benefits. Therefore, we get a representative sample and try to infer the underlying mechanics of the population that generate this property. Since we operate with a small sample, we also strive for characterising any uncertainty we have about the final statement.

How is the inference made in the example of the election poll? Let’s break it down into three steps:

Step 1: Define a population

This might sound easier than it actually is. In the simplest scenario the population of interest would be “all the people eligible for voting”. This definition is clear and it would even allow us to list all elements of the population (assuming we have census data at our disposal).

In reality, this is not the population we should care about, because not everyone votes (thankfully… or not). Therefore, to be more precise, we need to restate the definition to “all the people who will vote on the election day”. Unfortunately, it makes the inference much harder. How can we know who is going to vote? What are the reasons why people vote (or do not vote)? Is it even possible to tell it upfront?

We need to answer all these questions before we move on if we want to obtain high quality results. After all, this population is the group (and the only group) that affect the election results which is the outcome of our inference process.

Step 2: Describe a model of the population

Once we met and got to know the main character of the inference (the population), we need to find a representation of how its elements are related to each other. For example, we all know that similar people — people living in the same part of the country, having a similar job (or even similar brain structure [1]) are more likely to manifest similar political views.

If we are blind to these dependencies, it could skew our results significantly. It’s because without the right model it’s not possible to…

Step 3: Figure out the right sampling method

Polling is possibly one of the most complicated sampling tasks. There’s an infinite number of questions we can ask about its soundness. Is our sample large enough? The difference between candidates can be really small. Are we reaching all possible voters? Some voters could not be reachable. How do we even know that the sample data we collected are correct? Some people can lie to pollsters.

Worse still, once we assume the wrong population model, we end up with a completely biased sample. For example, it can be ok(ish) to approach a random person on the street and ask to share their preferences. However, it might be better to desist from asking their friends that are standing nearby (especially when they’re all waving red-black flags and calling out revolutionary slogans). If we couldn’t resist the temptation, it would not only skew the data, but also effectively decrease the size of the sample since we worked under the assumption that elements of the sample are independent, when in reality, they are not.

More on various types of biases you can read in this article.