The Probability Mass Function
By: Kanupriya Bhargava
Tags: Probability, Probability Theory, Mathematics, Data Science, Statistics

Probability Distributions

Probability distributions describe how probabilities are distributed over the possible values that a random variable can take. A random variable can be discrete- taking on only certain fixed values such as 0,1,2,… etc., or it can be continuous- taking any numerical values in an interval or collection of intervals.

In this article, I will focus on some frequently used discrete distributions, and attempt to explain the mathematical formulation of their Probability Mass Function, using simple examples.

The Probability Mass Function

The Probability Mass Function, or the PMF, provides the probability for each value of the random variable.

It is denoted by fₓ(x), where X is the random variable. Let us say that random variable X, takes an arbitrary value k. The PMF will give us the probability of that happening. Mathematically,

Properties of the PMF

(i) Positivity

A probability cannot be negative, and since fₓ(x) is a probability, it has to greater than or equal to zero.

(ii) Normalization

The sum of fₓ(x) over all possible outcomes k of the random variable is 1. This expression gives the probability of random variable X taking on any value, which is 1.

This concludes the not so interesting definition part! We are now equipped with the knowledge that we need to delve into the various distributions.

1. Bernoulli Distribution

The random variable X has a binary outcome. Let us denote these outcomes by 1 and 0. If the probability of X being 1 is p, then the probability of X being 0 is 1-p (Normalization property!).

A simple example of this is the tossing of a fair coin i.e there is a 50% chance of it landing heads up and a 50% chance of it landing tails up. Let 1 denote heads and 0 denote tails. Then the PMF can be described as-

2. Binomial Distribution

Suppose we toss a fair coin 3 times. Each toss is independent. What is the probability of getting exactly 1 head? There are 3 outcomes that give us exactly 1 head- (H, T, T), (T, H, T), and (T, T, H). Probability for this is

We can generalize this to tossing a coin n times. The probability of it landing heads up is p and of tails up is (1-p). What is the probability of getting exactly k heads? Where k=0,1,…,n

We will have k heads and (n-k) tails appearing in different sequences in the n tosses. Using combinatorics, we can work out how many such sequences exist. Then, we just need to multiply this to the probabilities for getting k heads and (n-k) tails. This will give us-

This expression is exactly the PMF of the Binomial Distribution!

This distribution gives the probability of obtaining exactly k success in n independent trials, where each trial has a binary outcome (Bernoulli Trial)- success with probability p, and failure with probability (1-p).

3. Geometric Distribution

Consider the following scenario- we toss a fair coin and it lands tails up. We toss is again and it lands tails up again. We keep tossing and it lands tails up 19 times in a row. Finally, on the 20th toss, it lands heads up. Assuming each toss is independent of the other, 19 tails in a row seems a bit unlikely, doesn’t it? But how unlikely (or likely) is it really?

We can answer these questions using the Geometric Distribution.

In Geometric Distribution, the random variable is the number of trials required to obtain success in independent Bernoulli trials (success or failure, 1 or 0, Heads or Tails, etc. ). The PMF is as follows-

Where k=1,2,3… is the number of trials before we see success and p is the probability of success.

For success to occur on the kᵗʰ trial with the probability p, there have to be (k-1) failures with probability (1-p). We simply multiply these probabilities and we have the PMF!

Let’s go back to the example. We have k=20 and p=0.5. So,

That is a very low probability! What if the coin was not fair? What if the probability of getting heads was only 30%? What is the probability of such a scenario then? Let’s calculate.

The probability is relatively high, but this scenario still seems very unlikely!

4. Negative Binomial Distribution

We are tossing a fair coin and suppose we have tossed it 9 times already. It has landed with heads up 4 times. What is the probability that we will get the 5th heads up on the 10th toss? This probability can be computed from the PMF of the Negative Binomial distribution.

In a series of independent Bernoulli trials, the random variable denotes the trial at which the nᵗʰ success occurs, i.e the nᵗʰ success occurs on the kᵗʰ trial.

If the nᵗʰ success occurs on the kᵗʰ trial, it means that in k-1 trials there have already been n-1 successes and k-n failures that occur in different sequences. We calculate the number of these sequences using combinatorics. We then simply multiply that number with the probability of getting n success and that of getting k-n failures. We denote the probability of success as p. Et voila, we have the PMF!

Let us calculate the probability for the example we discussed above. Here n=5 and k=10, and since we are using a fair coin, p=0.5 and (1-p)=0.5

5. Poisson Distribution

Poisson Distribution is one of the most frequently used discrete distributions. The random variable in this distribution counts the number of events in a given volume. For example, it can be the number of potholes in a 1 mile stretch of road, number of cars arriving at a toll booth in a 15-minute interval, etc.

The PMF, which gives the probability of observing k events in an interval, is given by the following expression-

Where k=0,1,2,… , e is the mathematical constant approximately equal to 2.718, and λ is the average number of events per interval (also called the event rate or parameter).

Let’s look at an example to understand this better.

Suppose we are interested in the number of arrivals at a drive-through during a 10-min period on a weekday morning. Historical data shows that the average number of cars arriving in a 10-min period is 6. We want to know the probability of there being 8 arrivals in a 10-min period.

Here, k=8 and λ=6, and we calculate the probability as-

The derivation for the PMF is a little math-intensive and could easily be a separate article, which is why I will not be going through it here. There are quite a few excellent articles that explain this in detail. I particularly liked this one.