Basics of Insurance Pricing. with a quick intro to GLM models | by Samuel Tober | Towards Data Science

In this article the basics of insurance pricing will be explained: Why pricing is so important, what happens if the pricing strategy is out of line, and how insurance companies set their premiums using statistical modelling. In the final section a very simple case study using real insurance data is presented to put some flesh on the bones of the ideas in the introduction.
The insurance industry is an industry concerned with hedging against the risk of uncertain financial loss, and the business of insurance companies is therefore largely a risk management endeavor. The insured trades future risk with an insurer for a fixed premium through a contract, known as the insurance policy, and if the policy holder is subject to a loss they can submit a claim to the insurer, if permitted by the policy. The premium is set by the insurer in advance of any claims, and hence it is vital for the company to predict the risks of their customers in order to set a profitable premium. With this in mind, it is not surprising that predictive modelling is extensively used in insurance companies; both in assessing customers and setting premiums.
As an example to highlight the importance of having a well thought-out pricing strategy, assume two insurance companies, A and B, exist and A has a low premium relative to the risk of loss, while B has an adequate premium in relation to the risk. In this scenario, a high risk customers would opt for A since their premium is relatively low compared to B, thus A would attract high risk customers and in effect see their margins being eaten up. On the contrary, if A’s premiums are too high they would not attract any profitable customers and still lose money. In the light of this simple example, we see why a competitive pricing strategy is paramount.
Moreover, according to the Accenture Financial Services’ 2017 Global Distribution & Marketing Consumer Survey, a study of 32,715 insurance customers across 18 markets, the price was found to be the number one driver of customer loyalty, with 52% of auto insurance customers choosing it as their top lever.
As explained above, the premium is set in relation to the risk of the customer, to ensure that the loss of the customer is covered. However, this does not account for all of the final price. Like any other business, an insurance company has their own costs and operate to make a profit. Therefore, the premium is set to not only cover the loss of the customer, but also to cover costs and keep a decent margin.
Nonetheless, the foundation of the premium is to choose a premium as per the risk of the customer. To do this, the risk of the customer seeking insurance, must be estimated. Depending on what insurance product we are regarding, different explanatory variables are used and commonly these are divided into two groups: Subject variables, concerning the person purchasing the insurance, and object variables, concerning the object to be insured; be it a house, a car or any other insurable object. For example, in home insurance, actuaries could use variables such as the live area of the property, the number of people that live in the household or the age of the person undertaking the insurance. The underlying assumption is that there is a correlation between these variables and the risk, what this correlation looks like of course differs, but as an example the risk of insuring a property usually decreases with the age of the undertaker as old age usually implies increased attentiveness and carefulness.
What is risk?
In insurance terms, risk is the chance that something harmful or unexpected could happen. This might involve loss, theft, or damage of valuable property and belongings, or it may involve someone being injured. From a statistical point-of-view risk can be defined as:
where L is the loss and e the period for which the insurance is valid (exposure). If we assume that the size of the claims is independent of the frequency of the claims we can make the following expansion:
where N is the number of claims, S the severity, or size of the claim and F the claim frequency.
Methods to Estimate Risk
It is no surprise that actuaries use statistical methods to estimate risk, until the 1980s actuaries relied on linear regression to model risk, but thanks to the establishment of a model known as the Generalized Linear Model (GLM), that changed. The advantage of the GLM over simple, linear regression is that the GLM model allows for a non-linear dependence through what’s called a link function (g in the equation below).
where Y is the dependent variable, X the independent variables and beta the parameters that are fitted through regression. Note that without the link function, the expression would just be simple linear regression.
Today, most insurance companies’ premiums are set using GLM models and they have to an extent become the comfort-zone of actuaries. However, in recent years, due to technological breakthroughs and the realization of big data in many industries, the popularity of using machine learning (ML) in business application has seen a surge, and the insurance industry is no exception. When employing ML models in insurance it is important to keep GDPR in mind as it puts demands on the interpretability of the models. Hence, easy to interpret tree-based models are usually favored, and even proven to boast better predictive accuracy than traditional GLMs on insurance data (See https://arxiv.org/abs/1904.10890).
Modelling Assumptions
In any model, assumptions must be made about the nature of the data and the variables used. We already encountered one assumptions that is common in insurance risk models, namely that the severity and frequency of a customers claims are independent. Using this assumption actuaries build two models, one for the severity and one for the frequency and subsequently combine the predictions from the two to find the risk estimate. However, this is not a necessary assumption as the risk also can be modeled directly. Which ever decision is made, the dependent variable, be it risk, severity or frequency, is always assumed to follow a known statistical distribution. For example the number of claims filed by a customer is usually assumed to be distributed according to a Poisson distribution, while the severity typically follows a gamma distribution. When actuaries model the risk directly the so called Tweedie distribution is a common choice.
Here an example of building a frequency model on home-insurance data using the statistics software R is presented.
First we have a look at the data at hand, the number of claims is our dependent variable, we visualize it through a histogram to see what distribution it follows and to get an overview of the problem:
Evidently, the number of claims is heavily skewed as most customers haven’t had any claims so we see a spike at zero. This motivates why using a Poisson distribution might be a good idea, by tuning the lambda parameter in the Poisson distribution this skewness can be captured.
The first step in modelling is to clean the data and choose what features to use. After this is done we can fit a GLM to the data.
First we read the data into R and split it into training/testing sets:
Frequency_data <- read.csv("C:/Users/Samuel Tober/Desktop/Projects/Data/Frequency_data.csv")set.seed(123) #Set seed for reproducibilitysample <- sample.split(Frequency_data, SplitRatio = 0.80) #Split data into 80% training 20% testtrain = subset(Frequency_data, sample==TRUE)test = subset(Frequency_data, sample==FALSE)
Then we fit a GLM model to the training data using the “glm” package:
model <- glm('NO_CLAIM_NOT_NULL ~ NO_INSUR + AGE_INSUR_PERS +                                    LIVE_AREA + offset(log(EXP_COV))', family = poisson(link = "log"),         data = train)
Here we model the number of claims (NO_CLAIM_NOT_NULL) as a function of the number of people in the household (NO_INSUR), the age of the customer (AGE_INSUR_PERS) and the live area of the property (LIVE_AREA). Note that we include the exposure (EXP_COV) as an offset in the GLM model, this is so that we can use Poisson regression for rate data (Poisson usually only applies to count data). Also, we specify link = “log”, where link refers to the link function, g, mentioned earlier, and we choose the logarithm since we are dealing with Poisson distributed data.
Next we test and evaluate the model using the mean squared error (MSE) and mean absolute error (MAE) as metrics (there are more suitable metrics such as the Poisson deviance, but here we choose a simple to understand metric for educational purposes, more about metrics in a future article):
pred <- exp(predict(model, test))MSE <- sqrt(mean((test$NO_CLAIM_NOT_NULL - pred*test$EXP_COV)^2)) #Mean squared errorMAE <- sum(abs(test$NO_CLAIM_NOT_NULL - pred*test$EXP_COV))/nrow(test) #Mean absolute error> MSE = 0.182480104096452> MAE = 0.0582112950956896
With this model we can now predict the claim frequency of an insurance customer, and if we similarly build a severity model, we arrive at a fully fledged pricing model under the frequency-severity independence assumption.
Do note that this is an extremely simplified example, please refer to https://www.casact.org/pubs/monographs/papers/05-Goldburd-Khare-Tevet.pdf and/or https://link.springer.com/book/10.1007/978-3-642-10791-7 for a more extensive explanations and showcase of the GLM model.