Maximum Likelihood Estimation and Poisson Regression | by Philip Wilkinson | Towards Data Science

In my previous article I introduced the basic concepts of probability and how this relates to maximum likelihood estimation and ordinary least squares regression. In this article, I will continue to build on this foundation but in the case of Poisson regression through its application to the gravity model.
In contrast to linear regression, which assumes that the dependent variable is normally distributed with constant variance, Poisson regression assumes that the target variable distribution has a Poisson distribution. Thus, the variance of the independent variable is expected to equal to its mean. This is used in the case where we are working with discrete, count based data, where the independent values are non-negative integers, and events are assumed to be independent of each other (although this may not actually be the case). This Poisson distribution can be represented by its probability mass function (a version of the probability density function but for discrete variables) which takes the form:
Translated into words, this simply means that the probability that a random variable Y takes the value of y_i, is a function of the mean of the distribution λ_i, and the number of counts of the event y_i.
This distribution can be modelled in python with the following code:
#import required librariesimport matplotlib.pyplot as pltimport numpy as np#create the subplotplt.subplots(figsize = (7,7))#plot the distributionsplt.hist(np.random.poisson(lam=0.5, size=3000))plt.xlabel("Random Poisson Distribution (0:3000, lambda = 0.5)", fontsize = 15)plt.ylabel("Count", fontsize = 15)plt.title("Random Poisson distribution", fontsize =20)plt.grid(True)
Whereby the resulting graph appears to be negatively skewed in comparison to what we would expect from a normal distribution and is especially prominent with small integer values.
The gravity model itself takes the form:
Which essentially says that the flows, T_ij, from i to j are proportional to the the population at the origin (V_i) and the population at the destination (W_j) and inversely proportional to the distance between them (d_ij). This is the basic form of the gravity model which is used to predict flows of population, goods, information and money between different geographic areas. This was adopted by analogy to Newton’s law of gravity as two bodies attracting each other in proportional to their size and inversely proportional to their distance.
In this case, k is a constant of proportionality which ensures that the predicted flows add up to the actual total flows, while the alpha, gamma and beta are all parameters to be estimated.
This model is used to predict the most likely value of T_ij, given the variables and parameters in the equation, meaning that the model becomes:
Where T_ij^hat is the predicted/expected value of the model. Thus, as before with normal regression, when we transform this into a linear format for regression purposes, we don’t transform the actual values but instead we transform the expected values given the data. Thus, this model becomes:
Where the right hand states that we are modelling the natural log of the expected values of the flows between i and j. This means that the expected value is related to the mean, λ_i, predicted by the model by a link function, which in the case of Poisson regression takes the form:
Where X represents a vector of all the i independent variables, β is a vector of all the parameters, and n is the number of observations that we have. The result of this is that we get the form:
It is this link function which ensures that λ is non-negative, even when the regressors, X, or the regression coefficients, β, have negative values [1].
Following the same line of thinking as we did for the linear relationship, the probability that each independent variable takes the observed values is given by:
Where y_n is our observed value for instance n, which in our case is T_ij, and X_n can be taken as a vector of all our dependent variables for instance n, which in our case is V_i, W_j and D_ij [2]. Since we have multiple observations, we want to maximise the likelihood of all of them occurring given a set of parameters. Since we are assuming that the instances are independent of one another, we thus want to use the joint probability relationship as the probability of each of them occurring:
This means that we get the value of our parameters that will make the given set of observed T_ij’s most likely.
Given that the multiplication of all of these values would be complicated, in order to simplify the mathematics we can take logs of both sides of the equation. Given that the natural log is a monotonic function (one y value for every x value) the log of the expected value should reach the maximum point at the same place as the value of the expected value. Thus, the complicated multiplication above becomes a simple summation:
Which, in the case of our gravity model, ultimately simplifies to:
whereby the λ_n is our predicted value and the T_ij is our actual value. This can then be differentiated by each parameter and solved to 0 in order to find the maximum likelihood estimation to get the functional form that best fits the data.
However, just like normally distributed maximum likelihood estimation, we can use regression, in the form of Poisson regression, to be able to approximate the solution [2]. This of course can be implemented in python through the statsmodels library. Thus, how the maximum likelihood estimation procedure relates to Poisson regression when the dependent variable is Poisson distributed.
[1] Lovett, A, Flowerdew, R 1989. Analysis of count data using Poisson Regression. The Professional Geography, 41,2, 190–198
[2] Fotheringham, S, Williams, P 1983. Further Discussion of the Poisson Interaction Model, Geographical Analysis, 15, 4, 343–347
If you liked my article and want to read more, feel free to use the following links: