Probability: classical, frequency-based and subjective approaches
By: Valentina Alto
Tags: Subjective Approach, Statistics, Gambling, Probability, Empirical Approach

Probability can be defined as a tool to manage uncertainty. Whenever an event is neither the certain one (with probability=1) nor the impossible one (probability=0), we are facing an uncertain situation, hence we need to attribute our event a likelihood of occurrence, which is, indeed, probability.

In this article, I’m going to present the three approaches to probability, which provide different interpretations of that concept and different assumptions to start with. Those approaches are:

Classical approach

Frequency-based (or empirical) approach

Subjective approach

Let’s dwell on all of them.

Classical approach

This approach traces back to the field where probability was first sistematically employed, which is gambling (flipping coins, tossing dice and so forth). Gambling problems are characterized by random experiments which have n possible outcomes, equally likely to occur. It means that none of them is more or less likely to occur than other ones, hence they are said to be in a symmetrical position.

The idea of the classical approach is that, given a collection of k elements out of n (where 0≤k≤n), the probability of occurrence of the event E represented by that collection is equal to:

To give you the intuition, let’s imagine you are tossing a dice and you want to predict the probability of the following collection of outcomes:

We know that the n possible outcomes are 6. The event “one” is 1 out of 6 outcomes, hence its probability is 1/6. Similarly, the event “five or six or one” (that is, the event in which one of those numbers turns out) represents 3 outcomes out of 6, hence the probability will be 3/6=0.5.

The classical approach is pretty intuitive, nevertheless it suffers from some pitfalls:

The assumption of symmetry is far too strong and irrealistic. Namely, imagine you want to know the probability of the event “tomorrow I will have a car accident”. The possible outcomes of this scenario are two: having a car accident or not having a car accident. Given that k=having a car accident, the probability of that event is 1/2, which, besides being a bit worrying, is not representative of the real likelihood of the event.

In this approach, there is no space for the concept of information, which is strictly related to probability. Let’s think about the previous example of the dice. Imagine you are told this dice is loaded and, instead of having the number “one”, it has two “six” (so the faces will be 2,3,4,5,6,6). Provided with this information, which probability would you attribute to the event “one”? Since it is impossible, the probability is equal to zero and not 1/6. Hence, probability does depend on the available information (the intuition will be clearer in the subjective approach)

Frequency-based (or empirical) approach

This approach was formally introduced in the field of natural science, where the assumption of symmetric position poorly fails. Instead, the idea on which this approach is based is that several experiments can be run under certain conditions considered as equivalent. Each experiment might lead either to success or to an insuccess. Hence, given n random experiments run under equivalent conditions, we define the frequency of “success” (which is an event E) as:

If we consider the “Empirical Law of Change”, which states that the more n increases, the more stable the frequency becomes, we can conclude that the limit of that frequency, for n->infinite, does exist and it is equal to the probability of the event “success”:

Let’s size the difference between the frequency-based and classical approach with the following example. Imagine you want to know the probability of the outcome of your tossed coin being “head”. You start with your classical approach: since the possible n outcomes are two (head or tail), the probability of “head” is 1/2=0.5.

Now you decide to follow the empirical approach, and you start tossing your coin several times, let’s say 100. Out of your attempts, you obtained 55 “head” and 45 “tail”. Hence, the frequency of the event “head” is 55/100=0.55, and it can approximate the probability of the event “head”.

As you can see, we obtained two different probabilities (0.5 vs o.55) for the same event. The key difference is the role of information: after 100 experiments, you gathered empirical evidence that “head” occurred more often than “tail”: it might be that your coin is not perfect, and you can incorporate this information while formulating your conclusions.

This approach is not lacking of criticisms though:

Again, there is one big assumption which is the convergence property of the frequency, whose limit might not exist

Repeating experiments under equivalent conditions might not be possible

There are events extremely rare, for which is impossible to run many simulations (think about extreme natural events like tsunami).

Subjective approach

Developed by probabilist B. de Finetti, this is the most intuitive definition of probability. Indeed, according to that approach, the probability of an event is the degree of belief a person attaches to that event, based on his/her available information. This reasoning holds only under the assumption of rationality, which assumes that people act coherently.

Let’s provide a more specific definition. Imagine a lottery where you can win an amount of money equal to S if event E occurs. To participate, you have to buy one ticket. Now, which is the price you would be willing to pay to participate in the lottery? If you indicate that price as π(E, S), the probability of event E is given by:

Imagine you want to predict the probability that your favorite football team will win the match tomorrow. You have the possibility to participate in a lottery where, if the team wins, you obtain a prize of 1000€, otherwise you gain nothing. Which is the price you would be willing to pay to participate? Let’s say you are very confident about your team capabilities and you are willing to pay 700€. Hence, the probability your team wins the match tomorrow is:

This last approach does not count serious criticisms, since it resolves some pitfalls of the previous approaches (like the impossibility of repeating experiments under equivalent conditions, because of the uniqueness of many events) and, at the same time, does not contrast with other theories. Indeed, the evaluator who has to decide the price of the lottery is not prevented from running experiments, compute the frequency of successes and use this information to propose a price.

Basically, what in other approaches was a rule, in the subjective approach is an option.