ASSESSING INDIVIDUAL UNEXPLAINED VARIATIONIN NON-LIFE INSURANCE
BY

OLA HÖSSJER, BENGT ERIKSSON, KAJSA JÄRNMALM
AND ESBJÖRN OHLSSON
ABSTRACT
We consider variation of observed claim frequencies in non-life insurance,
modeled by Poisson regression with overdispersion. In order to quantify how
much variation between insurance policies that is captured by the rating factors,
one may use the coefficient of determination, R2, the estimated proportion of
total variation explained by the model. We introduce a novel coefficient of individual determination (CID), which excludes noise variance and is definedas the estimated fraction of total individual variation explained by the model.
We argue that CID is a more relevant measure of explained variation than R2
for data with Poisson variation. We also generalize previously used estimates
and tests of overdispersion and introduce new coefficients of individual explained
and unexplained variance.
Application to a Swedish three year motor TPL data set reveals that only
0.5% of the total variation and 11% of the total individual variation is explainedby a model with seven rating factors, including interaction between sex and age.
Even though the amount of overdispersion is small (4.4% of the noise variance)
it is still highly significant. The coefficient of variation of explained and unexplained individual variation is 29% and 81% respectively.
KEYWORDS
Claim frequency variation, coefficient of determination, coefficient of individual
determination, unexplained individual variation, overdispersion, Poisson regression, rating factors.
1. INTRODUCTION
The idea behind modern non-life insurance rating is that each customer should
pay a premium as close as possible to the expected value of the cost that heor she causes the company. Consequently, the pure premium (the premium
without loading for expenses and cost of capital), should be close to the expected
Astin Bulletin 39(1), 249-273. doi: 10.2143/AST.39.1.2038064 © 2009 by Astin Bulletin. All rights reserved.

250

O. HOSSJER, B. ERIKSSON, K. JARNMALM AND E. OHLSSON

value of the claim cost for each insurance policy. In practice, the actuary tries
to fulfill this goal by finding rating factors that describe the variation in the
expected cost between the policies. These factors are chosen so that the actuarial model will capture as much as possible of the variation in expectation
between customers. On the other hand, the risk, i.e. the deviation of the claim
cost from its expectation, is of course transferred to the company – in particular, it is not the goal to reduce the variance of the claim cost to zero.
A tariff analysis is most often carried out with the aid of Generalized Linear
Models (GLMs), the theory of which is well summarized in McCullagh and
Nelder (1989). Application of GLMs to non-life insurance has been considered,among others, by Brockman and Wright (1992). The tariff analysis is usually
made separately for claim frequency and average claim severity, using multiplicative models (Jung, 1968). For GLMs, this corresponds to using a log-link
function.
In this paper we focus on claim frequency under a multiplicative model.
Let Yi be the observed claim frequency for policy i and let gji denote the pricerelativity for rating factor number j for this policy compared to a reference
policy, j = 1, 2, …, q. The claim frequency of the multiplicative model can thenbe written
li 0 E (Yi ) = l0 g1i g2i ··· gqi ,(1)

where l0 is li for the reference policy. The price relativities are connected to
the GLM regression parameters b = (b1, …, bp ) through the log-link,
li = exp( bT xi ),where xi = ( xi1, …, xip ) is a vector of 0-1 dummy variables (covariates) indicating which particular parameters that apply to policy i.
In practice, there is always some variation left above the multiplicativemodel: two policies in the same tariff cell, i.e. with the same values on the rating
factors, still have some residual difference in their expectation, unexplained by
the multiplicative model. Our aim here is to present measures of explained
and unexplained variation. This serves two purposes: (i) it is an aid in choosing rating factors for the model, cf. the use of R2 in linear regression; (ii) it gives
an indication of whether there is a need for experience rating (bonus/malus
systems) at the individual level or not.
Several authors have suggested the use of credibility models for so called
optimal bonus/malus rating, see Lemaire (1995) for an overview. As explainedin Ohlsson and Johansson (2006) and Ohlsson (2008), credibility models can beviewed as random effect models, in particular this is convenient in a GLM
context. The multiplicative model above then becomes, if Ui denotes the random
effect for contract i,
E(Yi |Ui ) = l0 g1i g2i ··· gqi Ui = li Ui .

(2)

ASSESSING INDIVIDUAL UNEXPLAINED VARIATION IN NON-LIFE INSURANCE

251

Here E (Ui ) = 1 and Var(Ui ) can be used as the basis for a measure of theamount of unexplained individual variation, as explained below. Without reference to GLMs, Bühlmann and Gisler (2005, Chapter 4.13) discuss similar
models under the name credibility models with “a priori differences”. In their
Chapter 9, Bühlman and Gisler (2005) also discuss evolutionary credibility
models, which allow the Ui for different observational years to have less than
the 100% correlation implicitly assumed above.
While the total number of claims is what drives the cost of the insurancecompany, its variation is not an appropriate starting point for measuring theperformance of the chosen rating factors. The rating factors determine how thetotal premium is distributed among the policy holders, but does not affect the
number of claims, the cost or premium income of the company directly in agiven portfolio of policies. The goal of a tariff analysis is not to reduce the costof the company, other things equal, but to get the right price on a competitive market. The latter will, of course, in the end increase the revenue of thecompany, while the wrong price will result in adverse selection of customers.
To this end we suggest taking an individual perspective when measuring theperformance of the chosen tariff, defining the total variation of a portfolio of
insurance policies as the average mean square error of prediction, where “average” refers to choosing a policy at random, see Section 2.2.
We use a decomposition of this total variation in the portfolio into threeparts: explained individual variation, unexplained individual variation andnoise. This is similar to a decomposition defined by Johnson and Hey (1971)
and Brockman and Wright (1992, Appendix D), who refer to explained andunexplained individual variation as between cell variance and within cell variance respectively. The coefficient of determination, R 2, is defined as the estimatedfraction of total variance explained by the model. However, the noise part of
the total variance, which is the Poisson variance in a model where there isnothing more to explain (Var(Ui ) = 0) can never be explained. This suggeststhat a more relevant index is the coefficient of individual determination (CID),
defined as the estimated proportion of the total individual variance explainedby the model. It excludes noise variance and is (close to) one if we manage toexplain (almost) all variation between policy means.
In non-life actuarial applications, the likelihood based deviance is often
employed for model selection. In the same spirit, coefficients of determination
may be defined using likelihood methods and deviance rather than variancedecompositions, see for instance Maddala (1983), Cox and Snell (1989, pp. 208209), Maggee (1990) and Nagelkerke (1991). However, we believe that a variancedecomposition of the response variable (in our case claim frequency) is of particular interest to the experimenter, providing an intuitive explanation of thefraction of total variation that can be explained.
It is also of interest to test whether there is more variation left to explain
or not. We present tests that generalize those of Venezian (1981, 1990) who only
considers the special case with no covariates and constant duration. The tests
might be used as an indication of the need for bonus/malus systems and/or a252

O. HOSSJER, B. ERIKSSON, K. JARNMALM AND E. OHLSSON

search for additional rating factors. We also present an estimate of the relative amount of overdispersion, f, that differs slightly from the traditional one,based on Pearson’s x 2-statistic in that policies are weighted based on time duration, not estimated claim frequency. We also define coefficients of variation for
the exlained and unexplained individual variation, as well as for the noise.
The paper is organized as follows. In Section 2 we define the model and variance decomposition in more detail. Parameter estimation is considered in Subsection 3.1, including definitions of R 2 and CID. Tests of excess variance are
discussed in Subsection 3.2 and our findings are applied to Motor TPL (Third
Party Liability) insurance in Section 4. We demonstrate, for a tariff with threeyear durations, that only 0.5% of the total variation (=R 2) and 11% of the total
individual variation (= CID) in claim frequencies is explained. The explained
and unexplained individual variation have coefficients of variation 29% and
81% respectively. Further discussion of the results is provided in Section 5 andmore technical details are gathered in the appendix.
2. VARIANCE DECOMPOSITION AND UNEXPLAINED INDIVIDUAL VARIATION
2.1. A Mixed Poisson Model
Consider a portfolio of n insurance policies. For i = 1, …, n, let Ni be theobserved number of claims during a period of time, ti , so that Yi = Ni /ti . It is
assumed that conditional on Ui , Ni follows a Poisson distribution with expectation ti li Ui , and so the unconditional distribution is a mixed Poisson distribution, i.e.
Ni ! Po(ti Li ),(3)

where Li = li Ui and li is given by (2). The Poisson model is frequently used in
non-life insurance, see for instance Chapter 2 of Beard et al. (1984).
The unexplained individual variation is captured by the random variable
Li – in Motor TPL insurance this variable can be said to capture the accident
proneness of the driver – with mean
E(Li ) = l i .

(4)

We assume a variance functiona

Var(Li ) = zl i ,(5)for the accident proneness for some a > 0 and z $ 0. When z = 0, (2)-(4) define
a generalized linear model with log link function.
For non-life insurance, a = 2 is most well known, since then Var (Ui ) = z
is independent of the covariate xi and z becomes the squared coefficient of

ASSESSING INDIVIDUAL UNEXPLAINED VARIATION IN NON-LIFE INSURANCE

253

variation of Li |xi , a parameter independent of the chosen unit of time. The
extension to time varying random effects (see the appendix) is also most naturalfor a = 2. When a = 1, z = Var(Li ) /E(Li ) is the relative increase of variancecaused by the overdispersion. We regard a as a constant and z as an unknown
parameter. However, to keep the variance function more flexible, we will not
restrict a in advance. See Pocock et al. (1981), Hinde (1982), Breslow (1984)
and Lawless (1987) for more details on parameter estimation and choice ofvariance functions for overdispersed Poisson regression.
2.2. Variance Decomposition
For our purpose of measuring explained and unexplained variation, we firstneed a measure of the total variability in the portfolio. As explained above, therelevant measure here is not the variance of the total cost for the company, butrather an average of the variance for the individuals. For relevance, the average
should be weighted with the time duration ti , so that a one-year policy has
the same impact as two half-year policies. Conceptually, this may be viewed as
if we drew a policy at random from the portfolio, giving each policy i a probability proportional to its ti . The mean claim frequency (of a randomly drawn
policy) is thenl = ! t i li / ! t i ,
i

(6)

i

n

where !i is short for ! i = 1.
The benchmark for measuring the effect of choosing a tariff should be a tariff
where all policies are assigned the average l. (Note that l differs from l0 in (1),which is the claim frequency of a reference policy, chosen to have a price relativity of one for all rating factors.)
The average mean squared error of prediction (AMSEP) for all {Yi } is2s 2 = ! t i E `^Yi - lh j / ! t i .
i

(7)

i

Now, since E(Yi | Li ) = Li and E(Li ) = li we have the simple decomposition
E((Yi – l)2) = (li – l)2 + E((Li – li ) 2) + E((Yi – Li )2). Hence, since E((Yi – Li ) 2 ) =E(Var(Yi |Li )) = li /ti , we can write s2 as a sum of three terms
s 2 = s12 + s22 + s32
2= !i t i ^li - lh / !i t i + z !i t i lia / !i t i + !i li / !i t i .

(8)

254

O. HOSSJER, B. ERIKSSON, K. JARNMALM AND E. OHLSSON

The first term of (8), s12, quantifies explained individual variation, the second
term s22 unexplained individual variation and the third term s32 represents noise,
i.e. the variance in a Poisson model without overdispersion. We will refer to
2

2
2
2
sun
exp = s 2 + s3 = ! t i E `^Yi - l i h j / ! t i
i

(9)

i

as the total unexplained variance. Following Venezian (1990), we also use theword excess variance for s22, since it quantifies the total excess of variance for
all Yi compared to what is expected under a pure Poisson model.
We have argued that the insurance company does not aim at predicting thecustomers’ Yi , but rather Li . Hence, it would be more relevant to consider the
AMSEP for {Li } rather than {Yi }, i.e.
2

2
sind
= ! t i E `^L i - lh j / ! t i ,
i

(10)

i

2
= s12 + s22 .
which could also be called the total individual variation; note that sindA variance decomposition similar to (8) is defined by Johnson and Hey(1971) and Brockman and Wright (1992) when a = 2. The difference is mainlythat they sum over tariff cells rather than policies and use a discrete approximation of accident proneness within each cell. With our approach we canhandle continuous as well as discrete covariates.
Traditionally, the total variance is decomposed into explained and unexplained variance components, and the explained variance is further divided
into various sources of variation. The special feature of (8) is that the unexplained variance is split into two terms representing individual variation andnoise. It is a special case of a more general variance decomposition introduced
by Hössjer (2008) for a large class of mixed regression models, including Poisson, logistic and linear regression.

2.3. Coefficients of Determination
To quantify the proportion of variance explained by the covariates, a traditional
R 2-type quantity would be the fraction of the total variation,
r =

s12
s2

,

while we have argued that it would be more relevant to use the fraction of thetotal individual variance,rind =

s12
2
sind

,

ASSESSING INDIVIDUAL UNEXPLAINED VARIATION IN NON-LIFE INSURANCE

255

2
which excludes the noise variance. Recall here that sind
= s12 + s22 .

The upper bound of rind is 1, corresponding to all relevant covariates being
used in the model. The upper bound 1 of r requires, in addition, that the noise
variance has been eliminated, which can only be achieved for very long time
durations, ti . Indeed, it is easy to see that rind is unaffected if all time durations
are, for instance, doubled, whereas r is increased.
2.4. Measures of Overdispersion and Coefficients of Variation
To assess the amount of unexplained variance several possible quantities could2,
be used, such as z or s22 . A more intuitive choice is perhaps 1 – rind = s22 / sindwhich gives the proportion of total individual variance not explained by thecovariates. Alternatively,
2
s un
exp

f =

s32

= 1+

s22

(11)

s32

quantifies, in relative terms, the amount of excess of the total unexplained
variance over the noise variance. A value larger than one indicates unexplained
individual variation. However, f shares the drawback of r in not being invariant with respect to magnified time durations. Alternatively we may use thecoefficient of unexplained individual variation
CUIV =s2=l2^f - 1h s3l2

ti / 1

=

f -1
l

(12)

as a measure of overdispersion. It quantifies the individual unexplained standard deviation in relation to the mean and is invariant with respect to magnified
time durations. We argue that CUIV it is a more intuitive and relevant measure of overdispersion than f for actuarial applications.
The coefficient of explained individual variation
s1
l

(13)

s3 t i / 1 1
=
,
l
l

(14)

CEIV =and the coefficient of noise variation
CNV =are two other quantities of interest. In fact, the variance decomposition (8) may
be restated by decomposing the squared coefficient of variation

256

O. HOSSJER, B. ERIKSSON, K. JARNMALM AND E. OHLSSON

CV2 =

s2
= CEIV2 + CUIV2 + CNV2
l2

into three sources of variation.
3. STATISTICAL INFERENCE
3.1. Parameter Estimation
The unknown parameters, a, b and z, can be estimated using full maximum
likelihood. This requires specification of the distribution of all Li and yields
quite complicated parameter estimates.
We will use a simpler approach, where first b is estimated separately by
maximum likelihood from a generalized linear model without overdispersion(z = 0). This facilitates use of standard software and moreover, it can be shownthat b is a consistent and asymptotically normal estimator of b even when z > 0,
see e.g. White (1982).
The next step is to estimate a, as explained in the appendix for the car accidents data set. Given b and a, we then estimate z by2!i b t i `Yi - l ij - l i lz =,!i t i l ia

(15)where li = exp(xi b T ) and a is regarded as known (for instance the estimateda, but not necessarily so). It is shown in Hössjer (2008) that asymptotically, inthe limit of large samples n, z has a normal distribution with mean z when ais regarded as a known constant. An explicit formula for the standard error isalso provided there.
The empirical version of the AMSEP for predicting claim frequency withthe constant l in (7) is, using (8),2!i t i `l i - lj ! i b t i `Yi - l ij - l i l ! i l i
s 2 = s12 + s 22 + s 33 =++!i t i!i t i!i t i ,2(16)

where l = !i ti li / !i ti . It gives rise to the coefficient of determinations12
R = r = 2,
s
2

(17)

Note that R 2 can be interpreted as the relative decrease in AMSEP we obtainfor {Yi } by going from a constant claim frequency l to a tariff of li’s, since its

ASSESSING INDIVIDUAL UNEXPLAINED VARIATION IN NON-LIFE INSURANCE

257

denominator is an estimate s 2 of the AMSEP with l and the nominator is s 22minus an estimate of the AMSEP with the li , i.e. of sunexp
in (9).
The empirical version of the more relevant AMSEP for prediction of Li in(10) is2= s12 + s22 .
sind

(18)

leading to an alternative to R 2 which we call the coefficient of individual determination
CID = rind =

s12
2 ,
s ind

(19)

respectively.
Like R2, CID can be interpreted as the relative decrease in AMSEP for {L i }2ofresulting from introducing varying li’s. The denominator is an estimate sind2
the AMSEP with l, see (10), while the numerator the difference of sind and an
estimator of the AMSEP with li , i.e. of
s22 = !i t i E ((L i – li )2 ) / !i t i .

(20)When all time durations are equal, the R 2 here is an analogue of the classicalR 2 used for (univariate) linear regression models, whereas CID has no suchanalogue. The reason is that that rind cannot be estimated for linear models,since the two components of the unexplained variance, s22 and s32 , cannot be
identified. On the contrary, CID is computable for both mixed Poisson and logistic regression models, as well as for multivariate linear regression models, see
Hössjer (2008), where also standard errors of both R 2 and CID are provided.
In order to estimate f from data, we use
2!i ti `Yi - l ij
.
f5 =
!i Yi

(21)

A slightly different version of f5 has !i li in the denominator instead. When
all time durations are equal, it follows from the GLM likelihood estimating
equations that the two versions are identical (see e.g. McCullagh and Nelder,1989). A formula for the standard error of f5 is provided in the appendix.
In the special case of constant time duration and no covariates (li / l), f5 isessentially the index of dispersion, i.e. the ratio of the sample variance andsample mean. In Section 5, we show that (21) is closely related to the Pearson
estimate f5 used in GLM theory.

258

O. HOSSJER, B. ERIKSSON, K. JARNMALM AND E. OHLSSON

The coefficients of variations (12)-(14) are estimated in the natural way, replacingsi2 by si2 and l by l = !i ti li / !i ti .
3.2. Testing Excess Variance
In order to test for excess variance, we formulate the null hypothesis H0 of noexcess variance against the alternative H1 of a positive excess variance, i.e.
H0 : z = 0,
H1 : z > 0,(22)which is equivalent to testing s22 = 0 against s22 > 0 or f = 1 against f > 1.
When the distribution of all Li is specified, one may employ a likelihood ratio
test to carry out (22). We will use a simpler approach based on excess variance,
which only involves the first two moments of Yi .
Our starting point is the excess variance statistic !i ti (Yi – li )2 – !i Yi , which
agrees with the numerator of (15), except that !i li is replaced by !i Yi . (Again,
the latter two sums are identical when all time durations are equal.) It is shownin the appendix that the standardized excess variance statistic
2!i ti `Yi - l ij - !i Yi,
T =2 !i l i2(23)

has an approximate standard normal distribution for large samples. Hence, atest with an approximate significance level 1 – a rejects H0 when T $ la , where
la is the (1 – a)-quantile of a standard normal distribution. Since T = c(f5 – 1),with c = !i Yi / 2 !i l 2i , we may also regard T as standardized version of f5 .
For constant time duration and no covariates, (23) amounts to testing overdispersion of stationary count data. Then the denominator of (23) simplifies
to 2n l. This test has been used by Venezian (1981, 1990) for car accident
data. An asymptotically equivalent approach, based on a x 2-approximation of!i (Yi – l)2, has been considered by Fisher (1950) and Rao and Chakravarti (1956).
4. CAR-ACCIDENT DATA
We will analyze Swedish car accident data from If P&C Insurance Company.
A detailed description of the data set can be found in Järnmalm (2006). Car
accidents are registered for customers having a uninterrupted 3 year period in
between January 1, 2002 and December 31, 2005. Shorter durations, in total
approximately 30% of the total portfolio are thus excluded. The rating factors

ASSESSING INDIVIDUAL UNEXPLAINED VARIATION IN NON-LIFE INSURANCE

259

TABLE 1
THE RATING FACTORS USED FOR THE CAR ACCIDENTS DATA SET.

Rating factor jClass

Variable

kj

Class Description

1: Customer years

1
2
3
4

0-2
3-5
6-10
11-

4

No. of years a customer has been insured in
the company.

2: Geographic zone

0-18

19

A division of Sweden into 19 geographicalzones.

3: Age of car

1
2
3
4
5
6

4: Premium class

0-9

10

Premium class is determined by type of car.

5: Driving distance

1-5

5

Five intervals of reported driving distances.
A larger class index corresponds to a longer
distance.

2

The sex of the customer.

13

The age of the customer.
The classes 4-12 have five year intervals, 3034, ..., 70-74.

0-2
3-5
6-8
9-12
13-16
17-

6: Sex7: Age

1
2
3

13

0-24
25-26
27-29

75-

6

are defined at the beginning of the risk period. The age depending factors, e.g.
Age of car, are for this reason not as accurate as possible, the advantage on
the other hand is that each individual’s characteristics are kept in one data
record. Hence, although our methodology in principle handles varying duration,
the present data set has ti / 1, measuring time in three year intervals. The sizeof the data set is n = 439 283, and customers report a total of 29 405 accidents
during the three year period.
The seven rating factors of the model are presented in Table 1. Although
our variance decomposition handles continuous covariates, we have followed
the current practice at If P&C and discretized the continuous covariates. Thisimplies that rating factor j is divided into kj classes. For j = 1, …, 5, each classwithin the given rating factor has a distinct regression coefficient br , except forthe class of the reference policy, which is chosen to have a fixed regression
coefficient, 0, not included in b. We model interaction between sex and age

260

O. HOSSJER, B. ERIKSSON, K. JARNMALM AND E. OHLSSONTABLE 2ESTIMATED RELATIVE INCREASE OF THE ACCIDENT RATE, exp(br ) FOR SELECTED RATING FACTOR CLASSES AND
CORRESPONDING WALD CONFIDENCE INTERVALS (CIS) WITH (APPROXIMATE) COVERAGE PROBABILITY 95%.
FOR EACH (COMBINED) RATING FACTOR, WE HAVE ONLY INCLUDED THE TWO CLASSES WITH MINIMALAND MAXIMAL exp( b r ). THE CIS ARE CALCULATED WITH THE HELP OF THE STANDARD SOFTWARE(PROC GENMOD IN SAS) FOR GLM LOGLINK POISSON REGRESSION ML-ESTIMATION. HENCE THE
OVERDISPERSION IS MODELED SLIGHTLY DIFFERENTLY THAN FOR THE MIXED POISSON DISTRIBUTION (3).
THIS DOES NOT CHANGE THE PARAMETER ESTIMATES b r , BUT THE CIS ARE SLIGHTLY AFFECTED.
THE DIFFERENCE IS HOWEVER NEGLIGIBLE, SINCE THE AMOUNT OF OVERDISPERSION z IS SMALL.

Rating factor
Intercept
Customer years
Geographic zone
Age of car
Premium class
Driving distanceSex/ageClass

exp ( b̂r )exp (Ib r )

1
4
2
16
2
6
1
6
1
5
Female/13
Female/10

0.05901.23641.00000.54751.0021
1.06410.72790.39881.58730.82031.25451.45930.8740(0.0470,0.0742)(1.1922,1.2822)(1.0000,1.0000)(0.5056,0.5925)(0.9468,1.0607)(1.0242,1.1055)(0.6811,0.7779)(0.2484,0.6371)(1.2815,1.9662)(0.7898,0.8520)(1.1739,1.3407)(1.3475,1.5471)(0.8125,0.9400)(rating factors 6 and 7), giving k6 k7 combined classes, of which one is chosen
as reference. The covariates are chosen as xi1 = 1 (the intercept) and, for r > 1,
xir = 1 if individual i belongs to the given (combined) class and 0 otherwise.
The total number of regression coefficients is5

p = 1 + ! _ kj - 1i + (k6 k7 – 1) = 65.
j =1Table 2 shows results of a standard GLM analysis, including parameter estimates br and the associated confidence intervals of some selected regression
coefficients. The average estimated claim frequency isn

l = n - 1 ! l i = n - 1 ! Yi = 0.0669
i

i =1

per three year intervals.
The next step is to test for overdispersion. For the data set analyzed by Venezian(1990), the overdispersion is highly significant. Our conclusion is the same,
since the test statistic for excess variance isASSESSING INDIVIDUAL UNEXPLAINED VARIATION IN NON-LIFE INSURANCE

T = 19.89,

261
(24)

so that the null hypothesis of no excess variance is rejected at level 0.001.
As a comparison, T = 23.30 for a model with no covariates. Hence the rating
factors only decrease the significance of excess variance marginally.
To assess more explicitly the impact of the rating factors, we estimated thethree components of the empirical variance decomposition (16) as
s12 = 0.0003732,
s22 = 0.0030,
s32 = l = 0.0669.
Inserting these values into (17) and (19), we get surprisingly low coefficientsof determination
R 2 = 0.0053,
CID = 0.1120.

(25)

Only about 0.5% of the total variation and 11% of the total individual variation is thus explained by the rating factors. In Figure 1 both R 2 and CID areplotted as functions of time, assuming all policies in the portfolio have thesame time duration t years. The two individual variances are constant, whereas
the noise variance s32 is inversely proportional to t. Hence R 2 increases witht whereas CID is constant. We notice that R 2 = 0.18% if t = 1 and that t = 60.2
is required in order for R 2 to reach 0.5CID. Of course, in practice, the time
duration t cannot be varied in this way. An insurance contract usually lasts
for one year. On the other hand, it is still of interest to consider claims over
several years, and then several policies may remain unchanged for at least, say,five years. In any case, Figure 1 illustrates that noise is by far the dominating
source of variation for time durations used in practice and that unrealistically long durations would be required in order to reduce noise variancesignificantly.
The relative excess variance is estimated as
f5 = 1.0442.
and the coefficients of variation as%
CEIV = 0.289%
CUIV = 0.813%
CNV = 3.866%
CV = 3.962.

(26)

262

O. HOSSJER, B. ERIKSSON, K. JARNMALM AND E. OHLSSONFIGURE 1: Plot of R2 (solid line) and CID (dotted line) versus t for the car accident data set,assuming all policies remain in the portfolio for t years, with s12 (t) = s12 , s22 (t) = s22 and s32 (t) = 3s32 /t.

Hence, the explained standard deviation is about 29% and the unexplained
individual standard deviation about 81% of the average claim frequency. The
noise coefficient of variation is much larger, 387%. As explained above, this isdue to the short time durations.
In order to give confidence intervals for (selected) parameter estimates, weneed to estimate the variance function, which requires estimation of a and z.
A regression method, explained in the appendix, yields(a,z0 ) = (1.3051, 0.0946).

(27)

For the final estimate (15) of the dispersion parameter z we use two different
values of a, which is regarded as known, and obtainz = *0.0442,

if a = 1,

0.0979,if a = 1.3.

Here a = 1.3 is taken from the initial regression analysis (27) and a = 1 is chosen to yield a simple overdispersion model.

ASSESSING INDIVIDUAL UNEXPLAINED VARIATION IN NON-LIFE INSURANCE

263

We report 95% Wald confidence intervals in Table 2 for selected regression
parameters and in Table 4 for r, rind, f and z. We notice that Ir , Irind and If
are very insensitive to the choice of a (1 or 1.3). This is due to the small amountof excess variance in the data, making the exact model of overdispersion lesscrucial. Hence we recommend using the simpler model with a = 1. Two versionsof If are reported based on standard errors defined in the appendix. The parametric model assumes a gamma distributed accident proneness, wheres thenonparametric model only includes the first four moments of Li | xi . They give
essentially the same confidence intervals.
5. DISCUSSION
In this paper, we have defined a general framework for quantifying explained
and unexplained variation of claim frequencies in non-life insurance, including
a new coefficient of determination, CID, generalizations of previously used
estimates of relative overdispersion (f5 ) and test statistics for overdispersion (T),as well as new coefficients of explained and unexplained individual variation,
CEIV and CUIV respectively.
Our purpose is not to solve new actuarial problems, but rather to provide
new, intuitive quantities that hopefully give new insights as regards to the quality of the chosen model. Our hope is that CID, CEIV and CUIV all become
valuable and much used tools when the actuary selects rating factors and classes
within each rating factor.
An application to a Swedish car accident data set reveals that the amount of
overdispersion is highly significant, but yet small in relation to noise variance.
This manifests itself by CID being much larger than R 2 and is explained by
the fact that time durations in Motor TPL are very short in relation to claim
frequencies. Similar analyses for other countries (see Järnmalm, 2006) show that
although the amount of overdispersion varies, it is persistently significant but
yet small in relation to noise variance.
Surprisingly, the proportion of explained variance is still very small after
removing noise variance. We obtained CID = 11.2%, whereas higher, but still
low values CID = 35.8% and CID = 31.2% can be deduced from the variancedecompositions of Johnson and Hey (1971) and Brockman and Wright (1992)
respectively. The low value of CID obtained for our data set and model may
have several reasons:1) The multiplicative risk assumption is only approximately correct. In particular, our model only includes interaction between two of the seven rating
factors in Table 1.
2) The number of classes within some rating factor could be increased or
replaced by continuous covariates.
3) The true claim frequencies li may be time varying, not constant. See theappendix for more discussion on this topic.

264

O. HOSSJER, B. ERIKSSON, K. JARNMALM AND E. OHLSSONTABLE 3MEAN AND EXCESS VARIANCE FOR PREMIUM GROUPS (SEE APPENDIX).

Ij

l̂ j

2
ŝ excess,j| Ij |

Accidents(0.000, 0.015)(0.015, 0.025)(0.025, 0.035)(0.035, 0.045)(0.045, 0.055)(0.055, 0.065)(0.065, 0.075)(0.075, 0.085)(0.085, 0.095)(0.095, 0.105)(0.105, 0.115)(0.115, 0.125)(0.125, 0.135)(0.135, 0.145)(0.145, 0.155)(0.155, 0.165)(0.165, 0.175)(0.175, 0.185)(0.185, 0.195)(0.195, 0.205)(0.205, 0.215)

0.01130.02000.03170.04100.05050.06000.06970.07960.0896
0.09930.10930.11930.12940.13940.14930.15900.16960.18060.18880.19880.2097–0.00142
0.000290.000910.001440.00132
0.00323
0.003450.00350
0.005410.001960.003270.00483–0.00430
0.009210.03867
0.14955
0.14359
0.06495
0.00247–0.15927–0.16571

1513
627
7462
35494
78455
100717
84785
57082
35490
20130
9942
4345
1818
836
391
126
42
15
8
3
2

13
13
226
1448
3933
6092
5945
4518
3239
1962
1057
517
220
122
61
23
9
5
2
0
0

TABLE 4
WALD CONFIDENCE INTERVALS Iq = (q – la /2 dq , q + la /2 dq ) OF VARIOUS PARAMETERS q.
THE ASYMPTOTIC COVERAGE PROBABILITY IS 95% (a = 0.05) AND dq IS THE STANDARD ERROR OF q.
THE ASSUMED a IS EITHER 1 OR 1.3 AND If IS EITHER NONPARAMETRIC (NP) OR PARAMETRIC (P).

q

a

Iq

r
r
rind
rind
f
f
f
z
z

1
1.3
1
1.3
1
1.3
1
1
1.3

(0.0049, 0.0058)(0.0049, 0.0058)(0.0967, 0.1274)(0.0967, 0.1274)(1.0383, 1.0500)NP(1.0383, 1.0500)NP(1.0384, 1.0500)P(0.0383, 0.0500)(0.0849, 0.1108)

ASSESSING INDIVIDUAL UNEXPLAINED VARIATION IN NON-LIFE INSURANCE

2654) A number of unknown individual characteristics are not included in themodel. For instance, the annual driving distance is self-reported and may
differ from the true one. Car drivers use different roads with varying risks,
and this variation is only to some extent captured by geographical zone.
The individual ability to drive safely is only to some extent explained by
sex/age. Other factors, such as psychological make-up and drinking habits,
cannot be included in the model.
5) Inclusion of customers with time duration less than three years in the portfolio may increase CID. These drivers typically have higher claim frequencies
than average.
Since individual variation of claim frequencies is very complex, we don’t statethat 1-5) are enough to guarantee a CID of 100%, simply that they to someextent explain the low CID found in our data set. For more discussion on thistheme we refer to Haight (2001), Lemaire (1995) and Brockman and Wright(1992).
Our work can be extended in several ways. A first extension is to consider
time varying covariates (see item 4 above) and random effects, as described in
more detail in the appendix.
A second extension is to use overdispersed Poisson distributions (ODP)
rather than mixed Poisson distributions. For ODPs, the parameter f is defined
directly in terms of the variance function;vi = Var(Yi ) = fli /ti ,(28)

for all policies i = 1, …, n, see McCullagh and Nelder (1989). In general, for
mixed Poisson distributions, (28) does not hold, and the definition of f in (11)cannot be reformulated in terms of the variance function of individual policies. An exception is a = 1 and ti / t, in which case (28) is satisfied for mixed
Poisson distributions as well, with f = 1 + zt.
Formally, the variance decompositions (8) and (16) can be defined for ODPs,
provided we change the interpretation of vi to that of (28). This in turn provides
us with R 2 and CID for ODPs. The interpretion of unexplained individual
variance and CID is less clear though, since ODP is not a mixed model, having
no random effects.
A third extension, when p/n is non-negligible, is to account for reduced degrees
of freedoms when defining R 2 and CID (see Hössjer, 2008), as well as f5 and T.
For our data set, this adjustment has a minor effect, since p/n = 1.48 · 10 –4.
A fourth extension is to replace ti by other weights wi when defining l, s 2,
the variance decomposition (8), r, rind and f. Various weighting schemes are
discussed in Hössjer (2008). One possibility is inverse variance weighting wi =ti / li . This choice of weights results in all policies having approximately the same
contribution to the unexplained part of s 2, sincewi Var(Yi ) . 1,

266

O. HOSSJER, B. ERIKSSON, K. JARNMALM AND E. OHLSSON

where the approximation is exact in absence of overdispersion. Since theseweights involve unknown parameters, we use estimated weights
wi = ti /li(29)

to compute l, s 2, the empirical variance decomposition (16), R 2, CID and f5 .
We may also generalize the version of T with !i li instead of !i Yi in the numerator to2

T =!i wi `Yi - l ij - !i wi `l i / t ij (29) x 2 - n
2 !i wi2 `l i / t i j

=

2

2n

= 19.864 ,

(30)

where
2

x = !
i

t i `Yi - l i j2li= 457 901

is the unscaled Pearson statistic (Pearson, 1900) for Poisson regression. The version of f5 with !i li in the denominator is generalized to!i wi `Yi - l ij
f5 =!i wi `l i / t ij

2
(29)

=

x2
n = 1.0424 ,(31)which agrees with the Pearson definition of f5 , except for using n instead of n – p
in the denominator. We notice that (30) and (31) only differ marginally from(24) and (26). Hence, for our data set, it seems that the choice of weights is
not crucial. This is probably due to the fact that all time durations are equaland the estimated claim frequencies li vary quite little for the majority of policies.
For other tariffs, this may not be the case and then it is of interest to compare
how various weighting schemes affect the coefficients of determination, test ofexcess variance, estimated overdispersion and coefficients of variation in termsof efficiency and power.
A fifth extension would be to include claim severity. Assuming Xij is thej th claim severity of the i th policy we may variance decompose the observed cost
rates
Ni

Z i = ! Xij / t i
j =1

ASSESSING INDIVIDUAL UNEXPLAINED VARIATION IN NON-LIFE INSURANCE

267

with weights wi = ti . An alternative approach is to treat claim severity separately
and condition on the observed Ni = ni . This leads to variance decompositionof the average claim costsni

Z i = ! Xij / ni ,j =1

for all policies with ni > 0, using weights wi = ni .

APPENDIX
Estimating the variance parameter a. We divide the estimated individual claim
frequencies li into 21 intervals (see Table 3), henceforth denoted as premium
groups. Let Ij be the j th premium group ( j = 1, …, 21) andl j = !i ! I l i / I j ,j22
sexcess,
j = !i ! I `Yi - l i j / I j - l jjthe estimated average premium and excess variance within I j . Assuming aa2
2
power relation E (sexcess,j ) = z l j , a weighted linear regression of log (sexcess, j )2against log(lj ) is employed, with weights proportional to | Ij |. Since sexcess, j isunreliable (and sometimes negative) for small premium groups, we only include
I3, …, I12 in the regression analysis, resulting in (27), where z0 is different from(15), which assumes a to be known. The estimate (27) is quite stable. Further
exclusion of i) I12 gives (a, z0 ) = (1.3234, 0.0997) and ii) I3 and I12 gives
2(a, z0) = (1.2971, 0.0930). In Figure 2, the pairs (lj , sexcess,j ), j = 3, …, 12 are plotted together with fitted variance curves based on (27) and a second curve witha = 1 and only z being estimated.
Asymptotic normality of z5 and the numerator of (23). Define
S = (S1, S2 ) = ( !i li , !i ti vi ) ,S = (S1, S2 ) = ( !i Yi , !i ti (Yi – li ) 2 ) ,where vi = Var(Yi ). We will prove that asymptotically, in the limit of large samples, S has a bivariate normal distribution with mean S and covariance matrixs! = e s 11
12

s12
.
s 22 o268

O. HOSSJER, B. ERIKSSON, K. JARNMALM AND E. OHLSSON

2FIGURE 2: Plot of s excess,j against lj for premium groups j = 3, …, 12, together with fitted variancecurves based on estimates (27) (dotted line), a = 1 and estimated z0 (solid line).

From this asymptotic normality of f5 follows. Indeed,
f5 = g( S ),f = g(S )where g(S1, S2) = S2 /S1. Let G = g(S) = (–f,1)/S1. Then, by Taylor expanding garound S, it follows that f5 is asymptotically normal with mean f and variance2

sf5 = G SG T = (s22 – 2fs12 + f2s11) / S12 .
Similarly, let C be the numerator of (23) and write
C = !i ( ti (Yi – li ) – Yi ) = k(S ),2

C = !i (ti vi – li ) = k(S ),(A.1)

ASSESSING INDIVIDUAL UNEXPLAINED VARIATION IN NON-LIFE INSURANCE

269

where k(S) = S2 – S1. Putting K = k(S) = (– 1, 1) we find that C is an asymptotically normal estimator of C with asymptotic variancesC2 = K SK T = s22 – 2s12 + s11 .

(A.2)

Following the lines of proof in Hössjer (2008), one verifies that
S = S + ! (Yi – li , ti (Yi – li )2 – ti vi ) + op (n1/2 ),

(A.3)

i

where the the last term is small in probability compared to n1/2 and hence
asymptotically negligible. A consequence of (A.3) is that the impact of replacing li by li in the definition of S has no effect on the asymptotic distribution.
It follows from (A.3) thats11 = !i vi ,s12 = !i ti ,(A.4)

s22 = !i k i ,where ti = ti E ((Yi – li )3 ) and ki = ti2E (((Yi – li )2 – vi )2 ). Inserting (A.4) into(A.1) and (A.2) we obtain
sf5 = S1– 2 !i ( ki – 2fti + f2 vi ) ,2

sC2 = !i ( ki – 2ti + vi ) .

(A.5)

To compute standard errors, we replace S1, f, vi , ti and ki by estimates andobtain2

d f5 = ( !iYi )–2!i ( ki – 2f5 ti + f5 2 (Yi – l i )2 ) ,d C2 = !i ( ki – 2ti + (Yi – l i )2 ),(A.6)

One option is to proceed nonparametrically and put
ti = ti ((Yi – l i )3 – vi (Yi – l i )),ki = ti2 ((Yi – l i ) – vi ) ,22

vi = li /ti + zl i .
a

We added the second term –vi (Yi – l i ) in the definition of ti in order to guaran2
tee that S is positive (semi)definite and thus d f5 and d C2 are non-negative.

270

O. HOSSJER, B. ERIKSSON, K. JARNMALM AND E. OHLSSON

Alternatively, a parametric approach is to assume a gamma distribution for all
Li . For instance, if a = 1, Li ! G(li /z, z), where G(a,b ) has density
f (x) =

1x a – 1e –x / b, x > 0.
b G]agaHence ti Li ! G(li /z, ti z ), and Ni = tiYi has a negative binomial distribution
Nbin(li /z,1 / (1+ ti z )). From moments of negative binomial distributions weobtainvi = ti–1li (1 + zti ),ti = ti–1li (1 + 3zti + 2z2 ti2 ),(A.7)

ki = 2li2 (1 + zti )2 + ti–1 li (1 + 7zti + 12z2 ti2 + 6z3 ti3 ),and their estimated analogues by plugging in li and z. Since z is often very small
for non-life insurance data the higher order powers of z make little contribution
to the standard errors. When z = 0, we obtain the denominator of (23) from
(A.5) and (A.7).
¡
Extending the variance decomposition to time-varying covariates and random
effects. For simplicity, assume that time is counted in units of years and that
all ti are integer valued and extend (2) and (3) tolij = exp( bTxij ),ti

Ni = ! j = 1Yi j ,(A.8)

Yij ! Po(lij Uij ),where lij , xij = (xij1, …, xijp ) and Uij is the time varying price relativity, covariate vector and random effect of policy i during year j respectively. We assume
that Yij and Ylk are conditionally independent given Uij and Ulk, that Uij and Ulk
are independent when i ! l and that
E (Uij ) = 1,
Cov(Uij ,Uik) = zlija /2 – 1lika/2 – 1 rk – j(A.9)

for some autocorrelation function r.
tt

Let li = ! ji = 1 lij /ti and Li = ! ji = 1 lijUij /ti be the average price relativity andaccident proneness of contract i. Then (A.9) gives an extensiona

Var (Li ) = zl i ,(A.10)

ASSESSING INDIVIDUAL UNEXPLAINED VARIATION IN NON-LIFE INSURANCE

271

of the variance function (5), whereti! lija /2 lika/2 rk – j .

al i = ti–2

j, k = 1

We notice that li # li when a # 2, with equality if and only if rk / 1 and either
a = 2 or lij / li. Hence, the variance of the averaged accident proneness is
reduced when the random effects or price relativities are time-varying.
There are at least two ways to extend the variance decomposition of Section 2. The first option is to retain (7) with Yi = Ni /ti , as before. This yields avariance decomposition for which the explained variance s12 and the noise variance s32 agree with (8), whereas the unexplained individual varianceas22 = z ! t i l i / ! t i .
iiis decreased as soon as li < li for at least one policy i . The conclusion is thatboth R2 and CID are increased when random effects within each policy are timevarying.
The second option is to consider the variance of the annual claim frequencies,
n

ti

2

s2 = ! ! E b_Yij - li l / ! t i .
i =1 j =1i

By similar calculations as in Section 2, this yields quite a different variancedecomposition with22

s12 = b !i t i ^li - lh + !i, j _lij - li i l / !i t i .
s22 = z !i, j lija / !i t i ,s32 = !i t i li / !i t i .
The noise variance s32 is increased for a variance decomposition based on
annual claim frequencies (by a factor t if ti / t) in agreement with the discussion in Section 4. The explained variance, s12, is enlarged as well, since we areable to explain not only li but also the variation of lij around li . The unexa
plained individual variance, s22 , also increases when a $ 1, since then !i, j l ij $a
!i ti li .
The conclusion is that in general, R 2 decreases if annual claim frequencies
Yij are considered rather than the averaged ones Yi , due to the increased noise
variance. On the other hand, CID may increase or decrease, depending on how

272

O. HOSSJER, B. ERIKSSON, K. JARNMALM AND E. OHLSSON

variable the price relativities and random effects are within each policy. If therandom effects are constant (rk / 1), then CID typically increases when annual
claim frequencies are considered.
ACKNOWLEDGEMENT
Ola Hössjer’s work was supported by the Swedish Research Council, contract
number 621-2005-2810. The authors wish to thank two anonymous referees,
whose comments significantly increased the quality of the manuscript.
REFERENCES
BEARD, R.E., PENTIKAINEN, T. and PESONEN, E. (1984) Risk Theory, The Stochastic Basis of
Insurance (3rd edition). Chapman and Hall.
BRESLOW, N.E. (1984) Extra-Poisson variation in log-linear models. Appl. Statist. 33(1), 38-44.
BROCKMAN, M.J. and WRIGHT, T.S. (1992) Statistical motor rating: Making effective use of yourdata. Journal of the Institute of Actuaries 119(111), 457-543.
BÜHLMANN, H. and GISLER, A. (2005) A Course in Credibility Theory and its Applications. Springer
Universitext.
COX, D.R. and SNELL, E.J. (1989) The analysis of binary data, 2nd ed., Chapman and Hall, London.
FISHER, R.A. (1950) The significance of deviations from expectation in a Poisson series. Biometrics 6, 17-24.
HAIGHT, F.A. (2001) Accident proneness: The history of an idea. Institute of Transportation
Studies, University of California, Irvine, USA.
HINDE, J. (1982) Compound Poisson regression models. In GLIM 82: Proceedings of the International Conference in Generalized Linear Models (R. Gilchrist, ed.), pp. 199-121, Springer, Berlin.
HÖSSJER, O. (2008) On the coefficient of determination for mixed regression models. Journal of
Statistical Planning and Inference 138, 3022-3038.
JOHNSON, P.D. and HEY, G.B. (1971) Statistical studies in motor insurance. Journal of the Institute of Actuaries 97, 199.
JUNG, J. (1968) On automobile insurance ratemaking. ASTIN Bulletin 5, 41.
Järnmalm, K. (2006) Measures of the remaining systematic variance between individuals when
divided into individual premium groups in non-life insurance. Master Thesis, Mathematical
Statistics, Stockholm University, Report 2006:15. (In Swedish.)
LAWLESS, J.F. (1987) Negative binomial and mixed Poisson regression. Canadian J. Statist. 15(3),209-225.
LEMAIRE, J. (1995) Bonus-Malus Systems in Automobile Insurance. Springer.
MADDALA, G.S. (1983) Limited-Dependent and Qualitative Variables in Econometrics. Cambridge
University Press.
MAGEE, L. (1990) R 2 measures based on Wald and likelihood ratio joint significance tests. Am.
Statistician 44, 250-253.
MCCULLAGH, P. and NELDER, J.A. (1989) Generalized Linear Models, second edition, Chapman
and Hall.
NAGELKERKE, N.J.D. (1991) A note on a general definition of the coefficient of determination.
Biometrika 78(3), 691-692.
OHLSSON, E. (2008) Combining generalized linear models and credibility models in practice.
Scandinavian Actuarial Journal 2008(4), 301-314.
OHLSSON, E. and JOHANSSON, B. (2006) Exact credibility and Tweedie models. ASTIN Bulletin36(1), 121-133.
PEARSON, K. (1900) On a criterion that a given system of deviations from the probable in caseof a correlated system of variables in such that it can be reasonably supposed to have arisen
from a random sampling. Phil. Mag. 50(5), 157-75.

ASSESSING INDIVIDUAL UNEXPLAINED VARIATION IN NON-LIFE INSURANCE

273

POCOCK, S.J., COOK, D.G. and BERESFORD, S.A.A. (1981) Regression of area mortality rates on
explanatory variables: what weighting is appropriate? Appl. Statist. 30, 286-295.
RAO, C.R. and CHAKRAVARTI, I.M. (1956) Some small sample tests for significance for a Poisson
distribution. Biometrics 12, 264-282.
VENEZIAN, E.C. (1981) Good drivers and bad drivers – a Markov model of accident proneness.
Proceedings of the Casualty Actuarial Society, LXVII, 65-85.
VENEZIAN, E.D. (1990) The distribution of automobile accidents – are relativities stable overtime? Proceedings of the Casualty Actuarial Society, LXXVII, 309-336.
WHITE, H. (1982) Maximum likelihood under misspecified models. Econometrica 50, 1-25.

OLA HÖSSJER (corresponding author)
Department of Mathematics
Stockholm University, S-106 91 Stockholm,
SwedenE-Mail: ola@math.su.se
Fax: +46-8-612 67 17,
URL: www2.math.su.se/~ola/