From Contemplative to Predictive Modeling(in actuarial science and risk management)
Arthur Charpentier, Florent Crouzet & Agathe Fernandes Machado
KU Leuven, 2024

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

1 / 135

Back in June 2015...
In June 2015, Jan invited me to give a Actuarial Contact Program (ACP) talk in
Leuven. The take-away slide started with

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

2 / 135

Back in June 2015...
In June 2015, Jan invited me to give a Actuarial Contact Program (ACP) talk in
Leuven. The take-away slide was claiming that data won’t speak for themselves

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

2 / 135

”End of Theory”... and Models ?

Source: https://www.wired.com/2008/06/pb-theory/
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

3 / 135

... Data can’t speak for itself

Source: https://marketoonist.com/2014/01/big-data.html
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

4 / 135

Arthur Charpentier Professor at Université du Québec à Montréal Denuit and Charpentier (2004, 2005) Mathématiques de l’Assurance Non-Vie, Charpentier (2014) Computational Actuarial Science with R, Bénéplanc et al. (2022) Manuel d’Assurance, Charpentier (2024) Insurance: Biases, Discrimination and Fairness.

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

5 / 135

Disclaimer“Whereof what’s past is prologue,” William Shakespeare (1610), The Tempest(that phrase stands for the idea that history sets the context for the present, see e.g.
Murray and Sinnreich (2006))
contemplation: noun, con·tem·pla·tion; the act of regarding steadily [MW]This is a long ongoing work, and, despite my efforts, it might contain errors of any type. Concepts andresults presented in those slides are probably either extremely vague, or wrong. All apologies.
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

6 / 135

Agenda“Rara avis in terris nigroque simillima cygno,” Decimus Iunius Iuvenalis (82 AD)“No amount of observations of white swans can allow the inference that all swans are white, but the observation of a single black swan is sufficient to refute that conclusion,” John Stuart Mill (1848)
Statistics rely (a lot) on i.i.d. (stationary) assumption
Machine learning focuses (mainly) on generalization
Accuracy is based on strong stability assumptions
Bias selection can impact interpretation of models
In many actuarial applications, we know that there are ruptures or changes...
what could we do about it ?
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

7 / 135

Agendawhat if we observe {x1 , · · · , xn } drawn under P but we need to computequantities under Q ?
what if we were able to estimate EP [Y |X] but we want EQ [Y |X] ?
what if X1 ̸⊥⊥ X2 under P, can we have X1 ⊥⊥ X2 under Q (fairness)
a lot of problems in actuarial science can be formalized like thatbrief introduction to ”transfer learning”
instance transfer (reweighting observations)feature transfer (mapping into a common space)parameter transfer

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

8 / 135

Climate, Finance and Insurance
As mentioned in Intergovernmental Panel on Climate Change, page 594“What does the accuracy of a climate model’s simulation of past orcontemporary climate say about the accuracy of its projections of climatechange? This question is just beginning to be addressed, exploiting the newly
available ensembles of models...” Randall et al. (2007)
A standard financial disclaimer, see e.g.,“Past performance is no guarantee of future returns,” Brain (2010)
or in insurance (about wildfire losses in California)“Looking backward has become less effective in predicting the future,” Frazier(2021)“History Doesn’t Repeat Itself, but It Often Rhymes,” Mark Twain (1874)
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

9 / 135

Motivation, statistics, rebus sic stantibus
Statistics : clausula rebus sic stantibus (”with things thus standing”)
Statistics commonly deals with random samples. A random sample can bethought of as a set of objects that are chosen randomly. More formally, it is”a sequence of independent, identically distributed random data points”. (...)
Independent and identically distributed random variables are often used as an
assumption, which tends to simplify the underlying mathematics. In practicalapplications of statistical modeling, however, the assumption may or may not berealistic ®
Let (Ω, F, P) denote a probability space,
Let y1 , y2 , · · · , yn be n i.i.d. samples of a random variable Y distributed by P

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

10 / 135

Motivation, statistics, rebus sic stantibus
An important concept in actuarial science is the return period.
“1.0.1. Conditions. The aim of a statistical theory of extreme values is toanalyze observed extremes and to forecast further extremes. (...) The essentialcondition in the analysis is the clausula rebus sic stantibus,” Emil Gumbel(1958), Statistics of Extremes, page 1.
rebus sic stantibus is Latin for ”with things thus standing” (”in gelijkblijvende
omstandigheden” or ”les choses demeurant en l’état”)
clausula rebus sic stantibus is the legal doctrine allowing for a contract or a treaty
to become inapplicable because of a fundamental change of circumstances,maxim omnis conventio intelligitur rebus sic stantibus for ”every convention isunderstood with circumstances as they stand”, by the Italian jurist Scipione
Gentili (1563–1616).
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

11 / 135

Motivation, statistics, rebus sic stantibus“The distribution from which the extremes have been drawn and its
parameters must remain constant in time (or space), or the influence that time(or space) exercises upon them must be taken into account or eliminated (...)
This assumption, made in most statistical work, is hardly ever realized.” Emil
Gumbel (1958), Statistics of Extremes, page 1.
“1.0.3. The Flood Problem. Similar stationary time series may easily beobtained for annual droughts, largest precipitations, snowfalls, maxima andminima of atmospheric pressures and temperatures, and other meteorologicalphenomena.” Emil Gumbel (1958), Statistics of Extremes, page 4.
Gumbel (1941a,b) discussed ”the return period of flood flows”, term used in Fuller(1914) Hazen (1930), on flood flows.

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

12 / 135

Motivation, statistics, rebus sic stantibus
Geometric distribution: The probability that the first occurrence of success requires
k independent trials, each with success probability p, the probability that the k-th trialis the first success isP(X = k) = (1 − p)k−1 p
for k = 1, 2, 3, 4, · · ·. And then, EP [X ] = p −1 .
Y1 Y2 Y3 Y4 Y5 Y6 Y7· · · Yk−1 Yk

1

1

1

1

1

1

1

···

1

1

0

0

0

0

0

0

0

···

0

0

Task Committee on Hydrology Handbook of Management Group D of ASCE (1996)

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

13 / 135

Motivation, statistics, rebus sic stantibus

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

14 / 135

Motivation, statistics, rebus sic stantibus
There is also a connection with the law of small numbers,
Law of small numbers: In a given period of n years, the probability of a given number
r of events of a return period µ is given by the binomial distribution as follows,!

P(X = r ) =n rµ (1 − µ)n−rr

and if n → ∞ and µ → 0 in such a way that nµ → λ then
!

n r
λr
µ (1 − µ)n−r → e −λ .
rr!

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

15 / 135

Motivation, statistics, rebus sic stantibus

If µ = 1/T , P no-occurrence in [0, t] = e −µt = e −t/T .
e −1 = 0.3678794 and 1 − e −1 = 0.6321206
This means, for example, that there is a 63.2% probability of a flood larger than
the 50-year return flood to occur within any period of 50 year ®
Is this only in the statistical literature ?
See ”generalization” in machine learning...

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

16 / 135(true) residuals

Motivation, statistics and overfit

In “old school econometrics”, consider model yi = x ⊤i β + εi , or y = Xβ + ε .
H
b = (X ⊤ X)−1 X ⊤ y so that yb = X(X ⊤ X)−1 X ⊤ yb = Xβ
Ordinary least squares, β(estimated) residuals(true) residualsb = (I − H)y = ( I − H ) ε .
Estimated residuals are ε(estimated) residuals(studentized) residuals
εbiStandardized residuals are bri = q.
b 1 − Hi,iσ(externally studentized) residuals(estimated) residuals

yi − yb(i)εbi=.
Externally studentized residual residuals are bti = qsiσ̃ 1 − Hi,i
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

17 / 135

Motivation, statistics and overfit
yb(i) is the predicted value for i-th point (i.e., x i ) when observation (x i , yi ) is removed
from the (training) dataset: leave-one-out cross-validation, or ”Jackknife”
Given a sample of size n, a jackknife estimator can be built by aggregating theparameter estimates from each subsample of size (n − 1) obtained by omitting
one observation. The jackknife technique was developed by Maurice Quenouille(1924–1973) from 1949 and refined in 1956. John Tukey expanded on the technique in 1958 and proposed the name ”jackknife” ®
Related to the idea of ”cross-validation”, Stone (1974)
Cross-validation is any of various similar model validation techniques for assessing
how the results of a statistical analysis will generalize to an independent data set(...) It is often used in settings where the goal is prediction, and one wants toestimate how accurately a predictive model will perform in practice. It can also
be used to assess the quality of a fitted model ®
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

18 / 135

Motivation, statistics and overfit”Under-fit & Over-fit” (and Goldilocks)

Plan (vanilla) GLM (logistic) regression.

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

19 / 135

Motivation, statistics and overfit”Under-fit & Over-fit” (and Goldilocks)

GAM (logistic) regression, with splines.

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

20 / 135

Motivation, statistics and overfit”Under-fit & Over-fit” (and Goldilocks)

k-nearest neighbors.

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

21 / 135

Motivation, statistics and overfit”Under-fit & Over-fit” (and Goldilocks)

k-nearest neighbors.

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

22 / 135

Motivation, statistics and overfit”Under-fit & Over-fit” (and Goldilocks)
Training error is shown in blue, validation error in red, both as a functionof the number of training cycles. If
the validation error increases (positive
slope) while the training error steadilydecreases (negative slope) then a situation of overfitting may have occurred.
The best predictive and fitted model
would be where the validation error has
its global minimum. ®

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

23 / 135

Motivation, statistics and overfit
In statistics and machine learning, the bias–variance tradeoff describes the relationship between a model’s complexity, the accuracy of its predictions, and how
well it can make predictions on previously unseen data that were not used totrain the model. In general, as we increase the number of tunable parameters ina model, it becomes more flexible, and can better fit a training data set. It issaid to have lower error, or bias. However, for more flexible models, there willtend to be greater variance to the model fit each time we take a set of samples
to create a new training data set. It is said that there is greater variance in themodel’s estimated parameters. ®
To compute biases and variances, we need some model...
Overfitting is more likely to be a serious concern when there islittle theory available to guide the analysis, ®

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

24 / 135

Motivation, machine learning, generalization
Machine learning : a key concept is generalization,“The generalization performance of a learning method relates to its prediction
capability on independent test data,” section 7.1 Hastie et al. (2009)
A central goal in designing a machine learning system is to guarantee that thelearning algorithm will generalize, or perform accurately on new examples after
being trained on a finite number of them. ®
Law of large numbers: If X , X1 , X2 , · · · , Xn , · · · are i.i.d. samples of a randomvariable distributed according to P, then for any (small) positive non-zero value ϵ > 0:
"

lim P

n→∞

 @freakonometrics § freakonometricsn
1X
EP [X ] −Xi > ϵ = 0
n i=1

#

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

25 / 135

Motivation, machine learning, generalization
To minimize the discrepancy between training and generalization errors, it is essential
to understand the implications of the law of large numbers.
This understanding is facilitated by concentration inequalities, which provide aquantitative measure of how much random variables deviate from their expected values.
Höffding’s inequality: If X , X1 , X2 , · · · , Xn , · · · are i.i.d. samples of a random variable
distributed according to P, such that P Xi ∈ [a, b] = 1, then for any (small) positive
non-zero value ϵ > 0:
"

P

n
−2nϵ2
1X
Xi > ϵ ≤ 2 exp
EP [X ] −n i=0(b − a)2#!

Note that we can writen1X
P  EP [X ] −Xi > (b − a)n i=0
 @freakonometrics § freakonometrics

s



−1
log(2δ) ≤ δ
2nfreakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

26 / 135

Motivation, machine learning, generalization
For binary and 0/1 loss,b n (m) ∼ √1
for a given m ∈ M, R(m) − Rnand we have the following ”worst case scenario”
sup

m∈M

n

o

b n (m) ∼R(m) − Rlog(Card(M))√
nsee Bousquet et al. (2003). And if M is infinite, see Vapnik-Chervonenkis (VC)
dimension, from Vapnik and Chervonenkis (1971).

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

27 / 135

Motivation, Time Series
On i.i.d. data, standard to use k-fold cross validation”This approach involves randomly dividing the set of observations into kgroups, or folds, of approximately equal size. The first fold is treated as avalidation set, and the method is fit on the remaining k-1 folds,” James et al.
(2013)

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

28 / 135

Motivation, Time Series
Standard approach, shuffle and then create k groups (sorted)”there is a bias-variance trade-off associated with the choice of k in k-foldcross-validation. Typically, given these considerations, one performs k-foldcross-validation using k = 5 or k = 10, as these values have been shown
empirically to yield test error rate estimates that suffer neither fromexcessively high bias nor from very high variance,” James et al. (2013)

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

29 / 135

Motivation, Time SeriesWith cross validation, it is possible to ”adapt” and take into account the dynamics,
for time series, use ”forward-validation”, Hjorth (1982, 1994), or ”rollingcross-validation, Bergmeir and Benı́tez (2012)

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

30 / 135

Motivation, actuarial science
Claims reserving : assume that dynamic of past payments
Astesan (1938)’s chain ladder• i accident year (cohort)• j development year
• i + j calendar year

Ci,j + Yi,j+1
Ci,j+1 = λj · Ci,jdevelopment factor

The primary underlying assumption of the chain-ladder method is that historicalloss development patterns are indicative of future loss development patterns. ®
Mack (1991)’s log-Poisson model, Yi,j ∼ P(µi,j ), µi,j = e ai +bj
Verrall (1996)’s additive version Yi,j ∼ P(µi,j ), µi,j = e a(i)+b(j) @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

31 / 135

Motivation, actuarial science
Observed incremental payment
Yi,j , where i + j ≤ n
Y1,0

Y1,1

Y1,n−2 Y1,n−1

Y2,0

Y2,1

Y2,n−2

Yn−1,0 Yn−1,1
Yn,0

Y1,0

Y1,1

Y1,n−2 Y1,n−1

Y2,0

Y2,1

Y2,n−2

Yn−1,0 Yn−1,1Yn,0Future incremental payment
Yi,j , where i + j > n, to be predicted @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

32 / 135

Motivation, actuarial science

accident year factor
Yi,j ∼ P(µi,j ), µi,j = e

ai + b j

development factor

a1

Y1,0

Y1,1

Y1,n−2 Y1,n−1

a2

Y2,0

Y2,1

Y2,n−2an−1 Yn−1,0 Yn−1,1an @freakonometrics § freakonometrics

Yn,0freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

33 / 135

Motivation, actuarial science
accident year factor
Yi,j ∼ P(µi,j ), µi,j = e

ai + b j

development factor
Once we have estimates, b
ai and bbj ,i, j = 1, · · · , n,bi,j = ebai +bjYb @freakonometrics § freakonometrics

b0

b1

bn−1

Y1,0

Y1,1

Y1,n−2 Y1,n−1

Y2,0

Y2,1

Y2,n−2bn

Yn−1,0 Yn−1,1Yn,0freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

34 / 135

Motivation, actuarial science
Taylor (1977)’s separation method“It is crucial to the logic underlying the chain-ladder method thatthe “exogeneous influences” should
not be too great (...) Clearly, it
would be preferable to separate, if
possible, the basic stationary claim
delay distribution from the exogeneous influences which are upsetting the stationarity”
Yi,j ∼ P(µi,j ), µi,j = e ai +γi+j

Y1,0

Y1,1

Y1,n−2 Y1,n−1

Y2,0

Y2,1

Y2,n−2

Yn−1,0 Yn−1,1γn−1 Yn,0γi+jbi,j = ebai +bY @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

35 / 135

Motivation, actuarial science

Taylor (1977)’s separation method
Yi,j ∼ P(µi,j ), µi,j = e ai +γi+j

Y1,0

Y1,1

Y1,n−2 Y1,n−1

Y2,0

Y2,1

Y2,n−2γi+jbi,j = ebai +b
Ywhat is γbi+j when i + j > n ?

Yn−1,0 Yn−1,1
Yn,0
γn+2

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

36 / 135

Motivation, actuarial science
Quarg and Mack (2004)’s Munich chain ladder, learn from both (accumulated)
payments and incurred estimates
C1,0

C1,1

C1,n−2 C1,n−1

I1,0

I1,1

I1,n−2 I1,n−1

C2,0

C2,1

C2,n−2I2,0I2,1I2,n−2

Cn−1,0 Cn−1,1

In−1,0 In−1,1

Cn,0

In,0

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

37 / 135

Motivation, actuarial science
Quarg and Mack (2004)’s Munich chain ladder, learn from both (cumulated)
payments and incurred estimates
C1,0

C1,1

C1,n−2 C1,n−1

I1,0

I1,1

I1,n−2 I1,n−1

C2,0

C2,1

C2,n−2I2,0I2,1I2,n−2

Cn−1,0 Cn−1,1

In−1,0 In−1,1

Cn,0

In,0

Constraint, Ci,n−1 = Ii,n−1 , ∀i

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

38 / 135

year tMotivation, actuarial science
Static life table :Denuit and Robert (2007) or
Pitacco et al. (2009)

The period life table represents
mortality rates during a specifictime period for a certain population ®

 @freakonometrics § freakonometrics

age x

Exposure Ex ,t
Deaths Dx ,t = Ex ,t − Ex +1,t+1
Dx ,t
= µx
At t, fixed qx =Ex ,t
Lx +1 = Lx · (1 − qx )freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

39 / 135

Motivation, actuarial science
Prospective (cohort) life table :Lee and Carter (1992) or
Pitacco et al. (2009)rate of change per age
log[µx ,t ] = ax + bx · κtaverage rate time indexµx ,t =

Dx ,t
Ex ,t − Ex +1,t+1
=
Ex ,t
Ex ,t

b t for t > today.
We need κ @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

40 / 135

Motivation, actuarial science
A cohort life table, often referred to as a generation life table, is used to represent
the overall mortality rates of a certain population’s entire lifetime. ®
Multiple prospective life table : Li and Lee (2005)
rate of change per age
rate of change per age
log[µix ,t ] = axi + bxi · κit + Bx · Kt
average rate time index time index“Mortality patterns and trajectories in closely related populations are likely to
be similar in some respects, and differences are unlikely to increase in the longrun. It should therefore be possible to improve the mortality forecasts forindividual countries by taking into account the patterns in a larger group,” Li
and Lee (2005)
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

41 / 135

Motivation, climate change

Climate, how to predict in ”uncharted territory”, Schmidt (2024)?
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

42 / 135

Motivation, climate change
A wildfire (or forest fire, bushfire) is an unplanned, uncontrolled and unpredictablefire in an area of combustible vegetation. ®
Climate risk in California (U.S.)“Why is it illegal in California to consider climate-informed catastrophe
models when setting wildfire insurance premiums?” Frazier (2021)
Some general context:
California Code Of Regulations, title 10, Chapter 5 (Insurance Commissioner), § 2644(”Determination of Reasonable Rates”)
Cal. Code Regs. tit. 10 § 2644.4 (Projected Losses)”Projected losses” means the insurer’s historic losses per exposure, adjusted bycatastrophe adjustment, as prescribed in section 2644.5.  @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

43 / 135

Motivation, climate changeCal. Code Regs. tit. 10 § 2644.5 (Catastrophe Adjustment)
In those insurance lines and coverages where catastrophes occur, thecatastrophic losses of any one accident year in the recorded period are replaced
by a loading based on a multi-year, long-term average of catastrophe claims.
The number of years over which the average shall be calculated shall be atleast 20 years for homeowners multiple peril fire, and at least 10 years forprivate passenger auto physical damage. Where the insurer does not haveenough years of data, the insurer’s data shall be supplemented by appropriate
data. The catastrophe adjustment shall reflect any changes between theinsurer’s historical and prospective exposure to catastrophe due to a change inthe mix of business. There shall be no catastrophe adjustment for private
passenger auto liability.  @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

44 / 135

Motivation, climate change
Climate risk in France,
Subsidence is a general term for downward vertical movement of the Earth’s
surface, which can be caused by both natural processes and human activities.
®
“To determine whether a drought episode is considered abnormal, the SWI
established for a given month is compared to the indicators for that samemonth over the previous 50 years. It is considered “abnormal” if the indicator
presents a return period greater or equal to 25 years,” Charpentier et al. (2022)1

1

not sure how to define properly a “25 year return period” for non-stationary time series,
Olsen et al. (1998), Salas and Obeysekera (2014), Read and Vogel (2015), Du et al. (2015)
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

45 / 135

Motivation, is-ought (and algorithmic fairness)
Fairness, of predictive models, Charpentier (2024),“is-ought” problem, David Hume (1739, 1748)
The is–ought problem arises when one makes claims about what ought to bethat are based solely on statements about what is (...) an ethical or judgmental
conclusion cannot be inferred from purely descriptive factual statements. ®
On a French motor dataset, average claim frequencies are 8.94% (men) 8.20% (women).
Logistic regression on k variables excluding gender.
k=0
k=2
k=8
k = 15
empirical
 @freakonometrics § freakonometrics

men
8.68%
8.85%
8.87%
8.94%
8.94%

women
8.68%
8.37%
8.33%
8.20%
8.20%

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

46 / 135

Motivation, is-ought (and algorithmic fairness)“Machine learning won’t give you anything like gender neutrality ‘for free’ thatyou didn’t explicitly ask for,” Kearns and Roth (2019).
What if distributions of scores, conditional on S1 or S2 (two protected attributes) aresignificantly different ? e.g., S1 ∈ {man, woman} and S2 ∈ {white, black}.

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

47 / 135

Motivation, is-ought (and algorithmic fairness)
Mitigation of discrimination (or unequal treatment) has a lot to do with the“is-ought” problem
Supreme Court Justice Harry Blackmun stated, in 1978, “in order to get beyond
racism, we must first take account of race. There is no other way. And in order
to treat some persons equally, we must treat them differently,” cited in Knowlton(1978), as mentioned in Lippert-Rasmussen (2020)
To quote another Supreme Court Justice, in 2007, John G. Roberts of the US
Supreme Court submits: “The way to stop discrimination on the basis of race
is to stop discriminating on the basis of race” Turner (2015) and Sabbagh (2007)

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

48 / 135

Selection Biais & Observational Data
Selection bias
The phrase ”selection bias” most often refers to the distortion of a statisticalanalysis, resulting from the method of collecting samples. If the selection bias is
not taken into account, then some conclusions of the study may be false. ®
Classical econometric problem, see Heckman (1974, 1976, 1979). Supposeyi = x ⊤i β + εibut either observed zi = 1, or not zi = 0. Suppose(zi =1 if zi⋆ = x ⊤i γ + ui > 0⋆
0 if zi = x ⊤i γ + ui ≤ 0

where U ∼ N (0, 1).
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

49 / 135

Selection Biais & Observational Datazi = 1 (not censored) when ui ≥ −x ⊤i γ, i.e.
⊤⊤
P Zi = 1 = P ui ≥ −x ⊤i γ = 1 − Φ(−x i γ) = Φ(x i γ).

Suppose
U
ε

!

!

∼N

0
1 ρ
,
0
ρ σ2

!!

i.e. E Yi Yi observed = E Yi Zi⋆ > 0 = E Yi Ui > −x ⊤
i γ ,







ϕ(x ⊤
i γ)
⊤
⊤ 
E Yi Yi observed = x ⊤
β
+
E
ε
U
>
−x
γ
=
x
β
+
ρσ
i
i
i
i
i
Φ(x ⊤
i γ)
If ρ = 0, no endogenous selection effects.

See Charpentier and Geoffard (2024) on bodily
injury claims, in France (trial or insurance).
 @freakonometrics § freakonometrics

inverse Mills ratio

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

50 / 135

Selection Biais & Observational Data
Propensity score
The “propensity” describes how likely a unit is to have been treated, given its
covariate values. The stronger the confounding of treatment and covariates, andhence the stronger the bias in the analysis of the naive treatment effect, thebetter the covariates predict whether a unit is treated or not. By having units
with similar propensity scores in both treatment and control, such confounding
is reduced. ®
”The propensity score is the conditional probability of assignment to aparticular treatment given a vector of observed covariates”, Rosenbaum andRubin (1983)
Suppose observed data are { x i , ai , yi }ni=1 drawn i.i.d (independent and identically
distributed) from unknown distribution P, where A ∈ {0, 1}, denotes either ”control”(placebo) or ”treated” (medicine).
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

51 / 135

Selection Biais & Observational Data
Let Y (a) (or Y (x , a)) denote ”potential outcomes” (under control and treatment).
In many application, the quantity of interest is TE (or TE (x )) the treatment effect,
TE = Y (1) − Y (0)
Name
1
2
3
4

Alex
Betty
Beatrix
Ahmad

Treatment
ai
0
1
1
0

Outcome (Weight)
yi yi (0) yi (1) TE
75
75
?
?
52
?
52
?
57
?
57
?
78
78
?
?

Gender
x1,i
H
F
F
H

Height
x2,i
172
161
163
183

···
···
···
···
···
···

Different notations are used y (1) and y (0) in Imbens and Rubin (2015), y 1 and y 0 in
Cunningham (2021), or yt=1 and yt=0 in Pearl and Mackenzie (2018).

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

52 / 135

Selection Biais & Observational Data
When ai = 1 is observed, and x i ,(observation : yi (1)counterfactual : yi (0)Following Holland (1986), given a ”treatment” T (here A), the average treatment
effect on outcome y is
τ = ATE = E Y (1) − Y (0) ,and following Wager and Athey (2018), given a treatment a, the conditional averagetreatment effect on outcome y , given some covariates x ,isτ (x) = CATE(x) = E Y (1) − Y (0) X = x .
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

53 / 135

Selection Biais & Observational Data
Strongly ignorable treatment assignment
Treatment assignment is said to be strongly ignorable if the potential outcomes are
independent of treatment (A) conditional on background variables X(Y (0), Y (1)) ⊥⊥A|X
Balancing score
Following Rubin (1973, 1974), a balancing score b(X) is a function of the observed
covariates X such that the conditional distribution of X given b(X) is the same fortreated (A = 1) and control (A = 0) unitsA⊥⊥ X | b(X)

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

54 / 135

Selection Biais & Observational Data
Propensity scoree(x) = P(A = 1|A = x)
As proved in Rosenbaum and Rubin (1983),
the propensity score e(x) is a balancing score
if treatment assignment is strongly ignorable given x then, it is also strongly
ignorable given any balancing function (specifically, given the propensity score)(Y (0), Y (1)) ⊥⊥ A | e(X).

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

55 / 135

Selection Biais & Observational Data
Horvitz -Thompson theory
One very early weighted estimator is the Horvitz–Thompson estimator of themean. When the sampling probability is known, from which the sampling population is drawn from the target population, then the inverse of this probability
is used to weight the observations. This approach has been generalized to many
aspects of statistics under various frameworks. In particular, there are weighted
likelihoods, weighted estimating equations, and weighted probability densities
from which a majority of statistics are derived. ®
Suppose observed data are { X i , Ai , Yi }ni=1 drawn i.i.d (independent and identically
distributed) from unknown distribution P, where A ∈ {0, 1}.
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

56 / 135

Selection Biais & Observational Data
Suppose observed data are { X i , Ai , Yi }ni=1 drawn i.i.d (independent and identically
distributed) from unknown distribution P, where A ∈ {0, 1}.
On can derive an Inverse Probability Weighted Estimator (IPWE )1A=a Y
P(A = a, X = x)where p(a|x) = P(A = a|X = x) =p(A = a|X)
P(X = x)estimate p(a|x) with p̂n (a|x), using any propensity model (e.g., logistic regression
model)µa = Eµ̂IPWE
=
a,n



n
1X
yi 1Ai =an i=1 p̂n (ai |x i ) @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

57 / 135

Selection Biais & Observational Data
We make the following assumptions.
(A1) Consistency: Y = Y (A)(A2) No un-measured confounders: {Y (0), Y (1)} ⊥⊥ A|X.
More formally, for each bounded and measurable functions f and g,
E(A,Y ) [f (Y (X, A)) g(A) | X] = EY [f (Y (X, A)) | X] · EA [g(A) | X] .
This means that treatment assignment is based solely on covariate data andindependent of potential outcomes.
(A3) Positivity: P(A = a|X = x) = EA [1(A = a) | X = x] > 0 for all a and x.

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

58 / 135

Selection Biais & Observational Data
from (A1)∗

E [Y (a)] = E(X ,Y ) [Y (X , a)] = E(X ,A,Y )E(X ,Y ) [Y (X , a)] = EX [EY [Y (X , a) | X ]] .



Y 1(A = a)
P(A = a|X )then simply (by (A3) EA [1(A = a) | X] > 0)
EY [Y (X, a) | X ] =E(A,Y ) [Y (X, a)1(A = a) | X]
EY [Y (X, a) | X ] EA [1(A = a) | X]=
EA [1(A = a) | X]
E[1(A = a) | X]

i.e.
EY [Y (X, a) | X] = E(A,Y )Y (X, a)1(A = a)X
E[1(A = a) | X]

The Inverse Probability Weighted Estimator (IPWE ) is known to be unstable if someestimated propensities are too close to 0 or 1 (see calibration issues).
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

59 / 135

Selection Biais & Observational Data
Augmented Inverse Probability Weighted Estimator (AIPWE ), Cao et al. (2009)
1µ̂AIPWE
=
a,n

n
X

n i=1

Yi 1Ai =a
1A =a − p̂n (Ai |Xi )− iQ̂n (Xi , a)p̂n (Ai |Xi )p̂n (Ai |Xi )!

n
1Ai =a1Ai =a1X
Yi + (1 −)Q̂n (Xi , a)=n i=1 p̂n (Ai |Xi )p̂n (Ai |Xi )!

nn
1Ai =a1X
1X
Yi − Q̂n (Xi , a)=Q̂n (Xi , a) +n i=1
n i=1 p̂n (Ai |Xi )!

!

here we need a regression estimator Q̂n (x, a) to predict outcome Y based on covariates
X and treatment A, for some subject i.
This approach is said to by ”doubly robust” (with a second order bias)
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

60 / 135

Post Stratification and Weights
Inspired from techniques used in sampling theory, use post-stratification techniques,
which is standard when dealing with a ”biased sample”.
The regression function is defined aµ(x) = EP [Y |X = x] = E EP [Y |X = x, A] =




Z
A

EP [Y |X = x, A = a]dP[A = a].

Following Moodie and Stephens (2022), the later can be writtenµ(x) =ZA

EP [Y · W |X = x, A = a]dP[A = a|X = x] = EP [Y · W |X = x],where W is a version of the Radon-Nikodym derivative
W =dP[A = a],dP[A = a|X = x]corresponding to the change of measure that will give independence between X and A.
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

61 / 135

Post Stratification and Weights
Properties of W
We have the following interesting property: let W be a version of the Radon-Nikodym
derivative
dP[A = a]W =,dP[A = a|X = x]then EP [W ] = 1, EP [A · W ] = EP [A] and EP [X · W ] = EP [X].
As proved in Fong et al. (2018),
EP [W ] =ZZ

w dP[A = a, X = x] =ZZ

w dP[A = a|X = x]dP[X = x]that can be written
EP [W ] =
 @freakonometrics § freakonometrics

ZZ

dP[A = a]dP[A = a|X = x]dP[X = x],
dP[A = a|X = x]freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

62 / 135

Post Stratification and Weights
and therefore
EP [W ] =ZZ

dP[A = a]dP[X = x] = 1.

Similarly
EP [A · W ] =ZZ

sw dP[A = a, X = x] =ZZ

sw dP[A = a|X = x]dP[X = x],andEP [A · W ] =ZZ

sdP[A = a]dP[X = x] =Z

EP [S]dP[X = x] = EP [S].

In statistics, this Radon-Nikodym derivative is related to the propensity score, as
discussed in Freedman and Berk (2008), Li and Li (2019) and Karimi et al. (2022).

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

63 / 135

Censoring, Kaplan-Meier and Reweighting
Classical problem in survival analysis, t is (true) failure times, a ∈ {0, 1} denotes
censoring, and c censored time.
ai = 1(ti ≤ ci ) and we observe yi = min{ti , ci }.
Let S(t) = P[T > t] and K (t) = P[C > t].
Observations are (yi , ai ). Suppose random censoring.
Following Kaplan and Meier (1958), recall that the survival function satisfies
S(t + 1) = q(t + 1) · S(t), for t ∈ N
S(t) = Prob(τ > t | τ > t − 1) Prob(τ > t − 1)
= (1 − Prob(τ ≤ t | τ > t − 1)) Prob(τ > t − 1)
= (1 − Prob(τ = t | τ ≥ t)) Prob(τ > t − 1)
= q(t)S(t − 1) ,
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

64 / 135

Censoring, Kaplan-Meier and Reweighting
If we iterate S(t) = q(t) · S(t − 1) = q(t) · q(t − 1) · S(t − 2) = · · · ,S(t) =tY

q(k) where q(k) = 1 − P[t = k|t ≥ k]

k=0
n
X
t
Y

dk
= 1 − i=1
qb(k) where qb(k) = 1 −KM (t) =
n
X
n
k
k=0

Sb

1(yi = k),1(yi ≥ k)i=1

d(k) is the number of known deaths at time k and n(k) is the number of those
persons who are alive (and not being censored) at time k − 1.
b (t).
Similarly, we can also derive the survival function for censoring times K , K
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

65 / 135

Censoring, Kaplan-Meier and Reweighting

Recall that without censoring, Fb (t) =n1X1(yi ≤ t) . With censoring, Robins andn i=1

Rotnitzky (1992) suggested
Fb (t) =nai 1(yi ≤ t)
1X
b (y − )n i=1 Kiwhere ai = 1(ti ≤ ci ). Satten and Datta (2001) proved that Fb (t) = 1 − SbKM (t).

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

66 / 135

Calibration (when ”probabilities” are badly assessed)
In many applications, we need to properly assess P(Y = 1| X = x)model calibration can be also used to refer to Bayesian inference about the valueof a model’s parameters, given some data set, or more generally to any typeof fitting of a statistical model. As Philip Dawid puts it, ”a forecaster is wellcalibrated if, for example, of those events to which he assigns a probability 30
percent, the long-run proportion that actually occurs turns out to be 30 percent.”®, see Dawid (1982).
b of Y is a well-calibrated prediction if E [Y |Y
b = p] = yb , for all p ∈ (0, 1).
Prediction Y
P“Out of all the times you said there was a 40 percent chance of rain, how often
did rain actually occur? If, over the long run, it really did rain about 40 percentof the time, that means your forecasts were well calibrated,” Silver (2012)“we desire that the estimated class probabilities are reflective of the true
underlying probability of the sample,” Kuhn and Johnson (2013)
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

67 / 135

Calibration (when ”probabilities” are badly assessed)“When we speak of the ‘probability of death’, the exact meaning of thisexpression can be defined in the following way only. We must not think of anindividual, but of a certain class as a whole, e.g., ‘all insured men forty-one
years old living in a given country and not engaged in certain dangerous
occupations’. A probability of death is attached to the class of men or toanother class that can be defined in a similar way. We can say nothing about
the probability of death of an individual even if we know his condition of lifeand health in detail. The phrase ‘probability of death’, when it refers to asingle person, has no meaning for us at all,” von Mises (1928, 1939).

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

68 / 135

Calibration (when ”probabilities” are badly assessed)
As explained in Van Calster et al. (2019), “among patientswith an estimated risk of 20%, we expect 20 in 100 tohave or to develop the event”.
If 40 out of 100 in this group are found to have thedisease, the risk is underestimated,
If we observe that in this group, 10 out of 100 have thedisease, we have overestimated the risk.
Hosmer-Lemeshow test, from Hosmer Jr et al. (2013) (logistic regression), and Bier score, from Brier (1950) and Murphy(1973).
Function plotted in psychological papers Keren (1991).

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

69 / 135

Calibration (when ”probabilities” are badly assessed)
BS =

n
2
1X
ŝ(x i ) − yin i=1

Calibration curve is defined as[0, 1] → [0, 1]
p 7→ g(p) := EP [Y | ŝ(x) = p](g:
Quantile Bins

Set ybi = ŝ(x i ), sorted yb1 ≤ yb2 ≤ · · · ≤ ybn , partition I1 , · · · , I10 of {1, 2, · · · , n}.
As in Pakdaman Naeini et al. (2015), consider scatter plot(u, vk ), where uk =1 X
1 X
ybi and vk =yi
nk i∈Ink i∈Ik @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)k

70 / 135

Calibration (when ”probabilities” are badly assessed)

Local Regression
Local regression of {(ŝ(x i ), yi )}gbα (p) =1 X
yi where I = {i : |ŝ(x i ) − p| ≤ α}.
nI i∈I

as suggested in Denuit et al. (2021)
One could also consider some kernel based local regression (of degree 1 or 2)

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

71 / 135

Calibration (when ”probabilities” are badly assessed)
Calibration scatterplot per quantile bins(see also Machado et al. (2024a,b))
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

72 / 135

Calibration (when ”probabilities” are badly assessed)
Local regression scatterplot per bins, [0; 0.1), [0.1; 0.2), [0.2; 0.3), [0.3; 0.4), etc

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

73 / 135

Calibration (when ”probabilities” are badly assessed)
Calibration scatterplot per local regression (small bandwidth) @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

74 / 135

Calibration (when ”probabilities” are badly assessed)
Local regression scatterplot per local regression (larger bandwidth) @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

75 / 135

Selection Bias and Importance Sampling
Importance sampling is a classical technique for Monte Carlo simulations.
Monte Carlo is based on the law of large numbers: if we can draw i.i.d. copies of arandom variable Xi ’s, under probability P, thenn1Xh(xi ) → EP [h(X )], as n → ∞.
n i=1Much more can be obtained, since the empirical distribution Pn (associated withsample {x1 , · · · , xn }) converges to P as n → ∞ (see, e.g., Van der Vaart (2000)).
Now, assume that we have an algorithm to draw efficiently i.i.d. copies of a randomvariable Xi ’s, under probability P, and we want to compute EQ [h(X )].
ndQ(xi )
1X
h(xi ) → EQ [h(X )], as n → ∞.
n i=1 dP(xi )| @freakonometrics § freakonometrics

{z
ωi

}

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

76 / 135

Selection Bias and Importance Sampling
The term is on the left is
bIS =
µ

n
1X
dQ(xi )
h(xi )
n i=1 dP(xi )and if the likelihood ratio is known only up to a multiplicative constant, define a“self-normalized importance sampling” estimate, as coined in Neddermeyer (2009) andOwen (2013),Pnωi h(xi )dQ(xi )
IS′b = i=1Pn.
µwith ωi ∝dP(xi )
i=1 ωibIS′ =µPnn1Xni=1 ωi h(xi )Pn=ωi h(Xi ) n→ EQ [X ], as n → ∞.
X
ω
n
i=1 i
i=1
ωi h(xi ){z}|→EP [X ]i=1|

 @freakonometrics § freakonometrics

{z

→1

}

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

77 / 135

Selection Bias and Importance Sampling
Law of large numbers: If X , X1 , X2 , · · · , Xn , · · · are i.i.d. samples of a randomvariable distributed according to P, then for any (small) positive non-zero value ϵ > 0,
if Q ≪ P:
"

lim Q

n→∞

 @freakonometrics § freakonometricsn1X
dQ(xi )EQ [X ] −ωi Xi > ϵ = 0, where ωi ∝n i=1
dP(xi )#

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

78 / 135

Selection Bias and Importance Sampling(1) suppose we can generate Poisson distribution P(8), we want some Poisson P(5),(2) suppose we can generate Poisson distribution P(5), we want some Poisson P(8).
In our context, one can define the ”importance sampling estimator” of EP [Y (1)], asbIS (Y (1)) =µ

1 X yi nt
1 X yi=,n1 t =1 e(x i ) nn t =1 e(x i )iiand a ”self-normalized importance sampling” estimate for EP [Y (1)] is
P

ωi yi
1
, where ωi =.
e(x i )
ti =1 ωib (Y (1)) = Pti =1µIS′

The ”self-normalized importance sampling” estimate for τ
IS′

τb

P

ti =1 ωi yi= P

ti =1 ωi

 @freakonometrics § freakonometricsPωi′ yi1
1−, where ωi =and ωi′ =.
′e(x i )
1 − e(x i )ti =0 ωi

− Pti =0

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

79 / 135

Distorting Scores to Mitigate Unfairness
When dealing with fairness and discrimination, Charpentier (2024), we want to insure,if yb = m(x, s), either
that model m satisfies the independence property if m(X, S) ⊥⊥ S , withrespect to the distribution P of the triplet (X, S, Y )demographic paritythat model satisfies the separation property if m(X, S) ⊥⊥ S | Y , with respect
to the distribution P of the triplet (X, S, Y )equalized odds
that model satisfies the sufficiency property if Y ⊥⊥ S | m(X, S) , with respect
to the distribution P of the triplet (X, S, Y )calibrationb ̸⊥b ⊥For demographic parity, maybe Y⊥ S under P, but Y⊥ S under Q.
Mitigation means distorting scores @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

80 / 135

Distorting Scores to Mitigate Unfairness
When dealing with fairness and discrimination, Charpentier (2024), we want to insure,b ⊥e.g., demographic parity, Y⊥ S (where S is a categorical sensitive attribute),weak version : E [Yb |S = A] = E [Yb |S = B]PP

L bb |S = A) =strong version : (Y(Y |S = B)

We need a ”distance” between two probability measures, PA and PB , e.g., Wasserstein,
Y ∼ PBY ∼ PBW22 (PA , PB ) =

min



E

T :[0,1]→[0,1]T (X ) − X

2 

=E



2
FB−1 ◦ FA (X ) − X

X ∼ PA @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)X ∼ PA81 / 135

Distorting Scores to Mitigate Unfairness
optimal transport mapping

quantile of level p in group B

T ⋆ (x ) = FB

−1

◦ FA (x )probability p associated with x in group A
T ⋆ is a monotonic (nondecreasing) mapping.
T⋆ =
i.e.

argmin EargminZ 1

T :[0,1]→[0,1] 0

T (X ) − X2 T (x ) − x

2dFA (x )where X ∼ FA ,T :[0,1]→[0,1]Y with Y ∼ FB

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

82 / 135hDistorting ScoresE

W2 (PA , PB ) =2

ZX

T ⋆ (X ) − X|FB−1 ◦ FA (x ) − x |2 dPA (x )

2 i

=

=

Z 1
0

min

E T (X ) − X2 

T :[0,1]→[0,1]|FB−1 (u) − FA−1 (u)|2 duMapping T is associated to a ”push-forward”
operator
PB (S) = T# PA (S) = P0 T −1 (S) , ∀S ⊂ R.
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

82 / 135

Distorting Scores
Optimal transport distance is interesting because it provides a constructive mapping,
point by point (as in the original Monge (1781)’s problem),

pile of sandhole / excavation site
Monge (1781), “Mémoire sur la théorie des déblais et des remblais ”
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

83 / 135

Distorting Scores
Optimal transport distance is interesting because it provides a constructive mapping,
point by point (as in the original Monge (1781)’s problem),

pile of sandhole / excavation site
Monge (1781), “Mémoire sur la théorie des déblais et des remblais ”
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

84 / 135

Distorting Scores
One can also consider barycenters of measures,Q⋆ = argmin
QX

ωj · W2 Q, Pj

2jas in Agueh and Carlier (2011). This optimal distribution Q⋆ is the distribution ofm⋆ (X, S)m⋆ (x, s = A) = P[S = A] · m(x, s = A)






+ P[S = B] · FB−1 ◦ FA m(x, s = A)m⋆ (x, s = B) = P[S = A] · FA−1 ◦ FB m(x, s = B)





+ P[S = B] · m(x, s = B).
p = FA m(x, s = A)score in group Am⋆ (x, s = A) = P[S = A] · m(x, s = A) + P[S = B] · FB−1 ◦ FA m(x, s = A)weights

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

85 / 135

Distorting Scores
For example, on distributions of scores, conditional on S1 and S2 ,See Charpentier (2024) for additional properties...

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

86 / 135

On Counterfactual Fairness“A decision satisfies counterfactual fairness if ‘had the protected attributes (e.g.,race) of the individual been different, other things being equal, the decision
would have remained the same’,” Kusner et al. (2017)
We achieve fairness on average treatment effect (counterfactual fairness on average)
ATE = E Y (A) − Y (B) = 0.


We achieve counterfactual fairness for an individual with characteristics x if
CATE(x) = E Y (A) − Y (B) X = x = 0.


 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

87 / 135

A little bit of geometry ?
Optimal transport exists in any dimension (but no intuitive results based on quantiles)
In geometry, a geodesic is a curve representing in some sense the shortest path(arc) between two points in a surface, or more generally in a Riemannian manifold(...) It is a generalization of the notion of a ”straight line”. ®
Formally, consider some metric space, (E , d). A constant speed geodesic between two
points x0 , x1 ∈ E is a continuous curve x : [0, 1] → E such that for every s, t ∈ (0, 1),d(xs , xt ) = |s − t|d(x0 , x1 ).
Given two measures P0 and P1 , and an optimal transport map T ⋆ such that
⋆ P , thenP1 = T#0
Pt = ((1 − t)I + tT ⋆ )# P0 , for any t ∈ (0, 1).
(that’s what we used with barycenters)
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

88 / 135

A little bit of geometry ?

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

89 / 135

A little bit of geometry ?

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

90 / 135

”Distances” between distributions (or measures)
Integral probability metrics (IPMs, Müller (1997)) are distances on the space ofdistributions over a set X , defined by a class F of real-valued functions on X as
DF (p, q) = sup E[f ( X )] − E[f ( Y )] .
f ∈F

X ∼p

Y ∼q

Discussed also in Dedecker and Merlevède (2007)
For two distributions p and q, the total variation distance (Jordan (1881); Rudin(1966)) between p and q isdTV (p, q) = sup |p(A) − q(A)| .
AEquivalently,

 @freakonometrics § freakonometrics

1
supdTV (p, q) =2 f :Rk →{0,1}

Z

f dp −

Z

f dq

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

91 / 135

”Distances” between distributions (or measures)(see e.g. https://djalil.chafai.net/blog/, with f : Rk → {−1, 1}, f = 1A − 1Ac )Thus, it is an IPM with F = {f : X → {0, 1}}, so that F is a set of indicatorfunctions for any event.
For two distributions p and q, Kolmorov-Smirnov distance (Kolmogorov (1933);
Smirnov (1948)) between p and q isdKS (p, q) = sup |p((−∞, t]) − q((−∞, t])| = sup |Fp (t) − Fq (t)| = ∥Fp − Fq ∥∞ ,


t∈R



t∈R

where Fp and Fq are the respective cumulative distribution functions.

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

92 / 135

”Distances” between distributions (or measures)
For two discrete distributions p and q, Kullback–Leibler divergence (Kullback andLeibler (1951) ) of p, with respect to q is
DKL (p∥q) =X

p(i) log

i

p(i)
,
q(i)

and for absolutely continuous distributions,
DKL (p∥q) =ZRp(x ) log

p(x )
dx or
q(x )

Z
Rk

p(x) log

p(x)
dx,
q(x)

in higher dimension.

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

93 / 135

”Distances” between distributions (or measures)
Consider two measures on p and q on R. Then define Cramér distance (Cramér(1928a,b) and Székely (2003))
Ck (p, q) =

Z ∞
−∞

|Fp (x ) − Fq (x )|k dx

1/k, for k ≥ 1

C2 is named ”energy-distance” in Székely (2003) and Rizzo and Székely (2016), and”continuous ranked probability score” in Gneiting et al. (2007).
It is an Integral Probability Metrics (IPM), sinceCk(p, q) =sup E[f ( X )] − E[f ( Y )] .
f ∈ Fk ′X ∼pY ∼qk −1 + k ′−1 = 1

where Fk ′ is the set of absolutely continuous functions such that ∥∇f ∥k ′ ≤ 1.
For example, if k = 1, ∥∇f ∥∞ ≤ 1 (corresponding to 1-Lipschitz functions).
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

94 / 135

”Distances” between distributions (or measures)
Consider two measures on p and q on R. Then define Wasserstein distance(Wasserstein (1969))
Wk (p, q) =

Z 1
0

|Fp−1 (u) − Fq−1 (u)|k du1/k, for k ≥ 1

Consider two measures on p and q on R.
W2 (p, q)2 =Z 1
0|Fp−1 (u) − Fq−1 (u)|2 du while C2 (p, q) =Z ∞−∞|Fp (x ) − Fq (x )|2 dx .

Consider two Gaussian distributions, then2

W2 N (µ1 , σ12 ), N (µ2 , σ 22 )= (µ1 − µ2 )2 + (σ1 − σ2 )2 ,and for two Bernoulli distributions, if p1 ≤ p2 , Wk B(p1 ), B(p2 ) = (p2 − p1 )1/k , √
W2 B(p1 ), B(p2 ) = p2 − p1 and W1 B(p1 ), B(p2 ) = p2 − p1 .
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

95 / 135

”Distances” between distributions (or measures)µ: multinomial distribution on {0, 1, 10}, with p = (.5, .1, .4)
νθ : binomial type distribution on {0, 10}, with q θ = (1 − θ, θ)Let θ⋆ = argmin{d(p, qθ )} or θ⋆ = argmin{d(p∥qθ )}with dKL (p∥qθ ), dJS (p, qθ ), dH (p, qθ ) and dHχ2 (p∥qθ ) @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

96 / 135

”Distances” between distributions (or measures)µ: multinomial distribution on {0, 1, 10}, with p = (.5, .1, .4)
νθ : binomial type distribution on {0, 10}, with q θ = (1 − θ, θ)Let θ⋆ = argmin{d(p, qθ )}with C1 (p, qθ ), C2 (p, qθ ), W1 (p, qθ ) and W2 (p, qθ ).
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

97 / 135

”Distances” between distributions (or measures)

W1 is an IPM where F the set of 1-Lipschitz functions, Kantorovich and Rubinstein(1958), i.e., if p and q have bounded support,
W1 (p, q) = sup
f ∈F

Z +∞
−∞



f (x ) d(p − q)(x ) ,F being the class of 1-Lipschitz functions
Gretton et al. (2012) introduced Maximum Mean Discrepancy (MMD) as a distance
between two measures (using RKHS representations).

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

98 / 135

Transfer learning in Machine Learning Literature

Source: Bozinovski and Fulgosi (1976), The influence of pattern similarity and transfer learning @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

99 / 135

Transfer learning in Machine Learning Literature
Framingham coronary heart disease (CHD) risk score, Wilson
et al. (1987, 1998); D’Agostino et al. (2001)
6 risk factors: age, BP, smoking, diabetes, total cholesterol (TC), andhigh-density lipoprotein cholesterol (HDL-C)
Framingham (U.S.) participants are of European descent
what if we use it on Chinese people ?, Liu et al. (2004)

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

100 / 135

Transfer learning in Machine Learning Literature
Framingham coronary heart disease (CHD) risk score, Liu et al.
(2004)
Refitted on Chinese population,
Chinese Multi-provincial Cohort Study (CMCS)

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

101 / 135

Transfer learning in Machine Learning Literature

Source: https://sefiks.com/2018/07/20/artistic-style-transfer-with-deep-learning/, ”learning style”
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

102 / 135

Quick Overview of ”Transfer Learning”
D = {(x i , yi )}

raw
data

Dt

b = argmin
θθ

training
data


X




i∈Dt

model

validationdata

evaluation
X

Dv




ℓ yi , mθ (x i ) + P(θ)
i∈Dvℓ yi , mbθ (x i )where formally, D is a a collection of i.i.d. observations from (X i , Yi ) ∼ P (or D ∼ P)
and we suppose that both Dt ∼ P and Dv ∼ P (concept of “generalization”)
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

103 / 135

Quick Overview of ”Transfer Learning”
More precisely, let us distinguish
the feature domain, Dx = (X , P), with X ⊂ Rk and P is a distribution on X ,we have a source (or training) and a target (or validation), and Ds and Dt
the task, (Y, m(·)), with Y ⊂ R is a label space and m is our predictive model,
m:X →Y
Transfer learning is about improving the target predictive model mt (·) by using Ds andms (·), even if we know that Ds ̸= Dt and ms (·) ̸= mt (·).
Xs ̸= Xt : heterogeneous transfer learning (different language)
Ps ̸= Pt : domain adaptation (different topic)
Ys ̸= Yt : possibly, {0, 1} vs. {0, N, 1}
Ps (y |x) ̸= Pt (y |x) (and therefore ms (·) ̸= mt (·))

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

104 / 135

Transfer learning and domain adaptation (Ps ̸= Pt )“Traditional machine learning is characterized by training data and testingdata having the same input feature space and the same data distribution.
When there is a difference in data distribution between the training data andtest data, the results of a predictive learner can be degraded,” Furht et al.
(2016)
notations
Consider some training (source) sample Ds = {(x s,i , ys,i )} and some test (target)
sample Dt = {(x t,i )}, both being i.i.d., with distributions Ps and Pt .
In a regression problem, y = m(x) + ε, i.e. m(x) = EP [Y |E = x]
Consider a parametric model, m(x|θ)), for some θ ∈ Θ.
Classical empirical risk minimization (ERM) leads to(b ∈ argmin
θθ∈Θ @freakonometrics § freakonometricsn
1Xℓ ys,i , m(x s,i |θ))
n i=1

)

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

105 / 135

Transfer learning and domain adaptation (Ps ̸= Pt )b is said to be consistent Shimodaira (2000). Otherwise...
If Ps = Pt , θImportance weighted empirical risk minimization (IWERM) is(θ̃ ∈ argmin
θ∈Θ

n
1X
Ps (x s,i )n i=1 Pt (x s,i )ℓ ys,i , m(x s,i |θ)) whichisnowconsistent.
One can define adaptative importance weighted empirical risk minimization (AIWERM)
θ̃ γ ∈ argmin


n
1 X

θ∈Θ  n i=1

Ps (x s,i )Pt (x s,i )

!γ



ℓ ys,i , m(x s,i |θ)) ,γ ∈ [0, 1] is the flattening parameter,(

 @freakonometrics § freakonometrics

γ = 0, ordinary ERM
γ = 1, IWERM

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

106 / 135

Transfer learning and domain adaptation (Ps ̸= Pt )One could consider regularlized importance weighted empirical risk minimization(RIWERM)(θ̃ λ ∈ argmin
θ∈Θ

n

1X
Ps (x s,i )ℓ ys,i , m(x s,i |θ)) + λP(θ)
n i=1 Pt (x s,i )),for some penalty function P(θ) (classically ∥θ∥ℓ1 (lasso) or ∥θ∥ℓ2 (ridge) types ofpenalty), and λ ≥ 0.

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

107 / 135

Transfer learning and domain adaptation (Ps ̸= Pt )Application in a regression context
Polynomial regression model,
Px ,θ ∼ N (Pβ (x ), σ 2 ) and θ = (β, σ 2 ), for some polynomial Pβi.e., y = β0 + β1 x + · · · + βk x k + ε where ε ∼ N (0, σ 2 ).
Suppose that the ”true” distribution isQx ∼ N (Q(x ), 1)
e.g., Q(x ) = −(2x − 1/2) + (2x − 1/2)3
Suppose also thatsource : πs ∼ B(as , bs )target : πt ∼ B(at , bt )(

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

108 / 135

Transfer learning and domain adaptation (Ps ̸= Pt )Linear model (mis-specified) and cubic model (well-specified)max log L(θ|y, x) = maxθ @freakonometrics § freakonometrics

θ

n
X
i=1

log p(y |x, θ) = minθnX(yi − Pβ(xi ) )2

i=1

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

109 / 135

Transfer learning and domain adaptation (Ps ̸= Pt )Linear model (mis-specified) and cubic model (well-specified)max log Lω (θ|y, x) = maxθ @freakonometrics § freakonometrics

θ

n
X
i=1

ω(xi ) log p(y |x, θ) = minθnXx at (1 − xi )bt
ix as (1 − xi )st
i=1 i

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)(yi −Pβ(xi ) )2

110 / 135

Transfer learning and domain adaptation (Ps ̸= Pt )Another example would be Pielke and Landsea (1998); Chavas et al. (2013); Weinkle
et al. (2018); Martinez (2020),“Both population and wealth have increased dramatically over the last several
decades and act to enhance the recent hurricane damages preferentially overthose occurring previously. More appropriate trends in the United States
hurricane damages can be calculated when a normalization of the damages aredone to take into account inflation and changes in coastal population andwealth,” Pielke and Landsea (1998)

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

111 / 135

Transfer learning and domain adaptation (Ps ̸= Pt )(Source: Pielke and Landsea (1998))

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

112 / 135

Transfer learning and heterogeneity (Xs ̸= Xt )Actuarial science is usually based on tabular data
Observation is (x i , yi )In many applications, use of ”demographic information”
An ecological fallacy is a formal fallacy in the interpretation of statistical data that occurs when inferences about
the nature of individuals are deduced from inferences
about the group to which those individuals belong ®
As in Goodman (1953, 1959).
See also Holt et al. (1996), Sedgwick (2015)(Source: https://www.towerhamlets.gov.uk/Documents/Borough stat...) @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

113 / 135

Transfer learning and heterogeneity (Xs ̸= Xt )Simpson’s paradox is a phenomenon in probability and statistics in which a trend
appears in several groups of data but disappears or reverses when the groups are
combined ®
Total
Top 6
A
B
C
D
E
F

Total
5233/12763 ∼ 41%
1745/4526 ∼ 39%
597/933 ∼ 64%
369/585 ∼ 63%
321/918 ∼ 35%
269/792 ∼ 34%
146/584 ∼ 25%
43/714 ∼ 6%

Men
3714/8442 ∼ 44%
1198/2691 ∼ 45%
512/825 ∼ 62%
353/560 ∼ 63%
120/325 ∼ 37%
138/417 ∼ 33%
53/191 ∼ 28%
22/373 ∼ 6%

Women
1512/4321 ∼ 35%
557/1835 ∼ 30%
89/108 ∼ 82%
17/ 25 ∼ 68%
202/593 ∼ 34%
131/375 ∼ 35%
94/393 ∼ 24%
24/341 ∼ 7%

Proportions
66%-34%59%-41%88%-12%96%- 4%
35%-65%53%-47%
33%-67%52%-48%

Data from Bickel et al. (1975) (discussed as an illustration of ”Simpson’s paradox”)
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

114 / 135

Transfer learning and heterogeneity (Xs ̸= Xt )sensitive

sensitive

P[ Y = yes | S = men ] ≥ P[ Y = yes | S = women ]overall admission
P[ Y = yes | X = x , S = men ] ≤ P[ Y = yes | X = x , S = women ], ∀x .
conditional on program“the bias in the aggregated data stems not from any pattern of discriminationon the part of admissions committees, which seems quite fair on the whole, but
apparently from prior screening at earlier levels of the educational system.
Women are shunted by their socialization and education toward fields ofgraduate study that are generally more crowded, less productive of completeddegrees, and less well funded, and that frequently offer poorer professional
employment prospects,” Bickel et al. (1975)
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

115 / 135

Transfer learning and heterogeneity (Xs ̸= Xt )Consider the following mortality rates in two hospitals (fake data)
Hospital A
Hospital B

Total
800/1000 = 80%
900/1000 = 90%

Healthy590/600 ∼ 98%
870/900 ∼ 97%

Pre-condition
210/400 ∼ 53%
30/100 ∼ 30%

Proportions
60%-40%90%-10%

There is no mathematical ”paradox”, per se.
We could have
A
a
C
c
≥ and
≥
B
b
D
d
and at the same time
A+C
a+c
≤
B+D
b+d
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

116 / 135

Transfer learning and heterogeneity (Xs ̸= Xt )Overall mortality rate for women, 8.12‰ in Costa Rica, against 9.29‰ in Sweden.
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

117 / 135

Transfer learning and heterogeneity (Xs ̸= Xt )September 27, 2023, the Colorado Division of Insurance exposed a new proposed regulation entitled Concerning Quantitative Testing of External Consumer Data and Information Sources, Algorithms, and Predictive Models Used for Life Insurance Underwriting
for Unfairly Discriminatory Outcomes– Section 5 (Estimating Race and Ethnicity) –Insurers shall estimate the race or ethnicity of all proposed insureds that haveapplied for coverage on or after the insurer’s initial adoption of the use of
ECDIS, or algorithms and predictive models that use ECDIS, including a third
party acting on behalf of the insurer that used ECDIS, or algorithms and predictive models that used ECDIS, in the underwriting decision-making process,by utilizing: BIFSG and the insureds’ or proposed insureds’ name and geolocation (...)
Bayesian Improved First Name Surname Geocoding, or “BIFSG”
External Consumer Data and Information Source, or “ECDIS”
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

118 / 135

Transfert learning and actuarial science, wrap-up
Finance“Past performance is no guarantee of future returns,” Brain (2010)
Climate models“The common investment advice that ‘past performance is no guarantee of
future returns’ and to ‘own a portfolio’ appears also to be relevant to climate
projections,” Reifen and Toumi (2009)
Use of proxies“In practice, we often have limited data on the true predictive task of interest,and must instead rely on more abundant data on a closely-related proxypredictive task (...) hospitals often rely on medical risk scores trained on adifferent patient population (proxy) rather than their own patient population(true cohort of interest) to assign interventions. Yet, not accounting for thebias in the proxy can lead to sub-optimal decisions.” Bastani (2021)
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

119 / 135

References
Adjaye-Gbewonyo, D., Bednarczyk, R. A., Davis, R. L., and Omer, S. B. (2014). Using the bayesian
improved surname geocoding method (bisg) to create a working classification of race and ethnicity
in a diverse managed care population: a validation study. Health services research, 49(1):268–283.
Agueh, M. and Carlier, G. (2011). Barycenters in the wasserstein space. SIAM Journal on
Mathematical Analysis, 43(2):904–924.
Astesan, E. (1938). Les réserves techniques des sociétés d’assurances contre les accidents
d’automobiles. Collection d’études sur le droit des assurances.
Bastani, H. (2021). Predicting with proxies: Transfer learning in high dimension. Management
Science, 67(5):2964–2984.
Bergmeir, C. and Benı́tez, J. M. (2012). On the use of cross-validation for time series predictor
evaluation. Information Sciences, 191:192–213.
Bickel, P. J., Hammel, E. A., and O’Connell, J. W. (1975). Sex bias in graduate admissions: Data
from Berkeley. Science, 187(4175):398–404.
Bousquet, O., Boucheron, S., and Lugosi, G. (2003). Introduction to statistical learning theory. In
Summer school on machine learning, pages 169–207. Springer.

 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

120 / 135

References
Bozinovski, S. and Fulgosi, A. (1976). The influence of pattern similarity and transfer learning upon
training of a base perceptron b2. In Proceedings of Symposium Informatica, volume 3, pages
121–126.
Brain, J. (2010). “past performance is not necessarily indicative of future results”—the proven-in-use
argument and the retrospective application of modern standards. In 5th IET International
Conference on System Safety 2010, pages 1–4. IET.
Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly weather
review, 78(1):1–3.
Bénéplanc, G., Charpentier, A., and Thourot, P. (2022). Manuel d’assurance. Presses Universitaires deFrance.
Cao, W., Tsiatis, A. A., and Davidian, M. (2009). Improving efficiency and robustness of the doubly
robust estimator for a population mean with incomplete data. Biometrika, 96(3):723–734.
Charpentier, A. (2014). Computational actuarial science with R. CRC press.
Charpentier, A. (2024). Insurance: biases, discrimination and fairness. Springer Verlag.
Charpentier, A. and Geoffard, P. (2024). To sue or not to sue. in progress.
Charpentier, A., James, M., and Ali, H. (2022). Predicting Drought and Subsidence Risks in France.
Natural Hazards and Earth System Sciences.
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

121 / 135

References
Chavas, D., Yonekura, E., Karamperidou, C., Cavanaugh, N., and Serafin, K. (2013). Us hurricanes
and economic damage: Extreme value perspective. Natural Hazards Review, 14(4):237–246.
Cramér, H. (1928a). On the composition of elementary errors: First paper: Mathematical deductions.
Scandinavian Actuarial Journal, 1928(1):13–74.
Cramér, H. (1928b). On the composition of elementary errors: second paper: statistical applications.
Scandinavian Actuarial Journal, 1928(1):141–180.
Cunningham, S. (2021). Causal inference. Yale University Press.
D’Agostino, R. B., Grundy, S., Sullivan, L. M., Wilson, P., Group, C. R. P., et al. (2001). Validation of
the framingham coronary heart disease prediction scores: results of a multiple ethnic groups
investigation. Journal of the American Medical Association, 286(2):180–187.
Dawid, A. P. (1982). The well-calibrated bayesian. Journal of the American Statistical Association,
77(379):605–610.
Dedecker, J. and Merlevède, F. (2007). The empirical distribution function for dependent variables:
asymptotic and nonasymptotic results in. ESAIM: Probability and Statistics, 11:102–114.
Denuit, M. and Charpentier, A. (2004). Mathématiques de l’assurance non-vie: Tome I Principes
fondamentaux de théorie du risque. Economica.
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

122 / 135

References
Denuit, M. and Charpentier, A. (2005). Mathématiques de l’assurance non-vie: Tome II Tarification etprovisionnement. Economica.
Denuit, M., Charpentier, A., and Trufin, J. (2021). Autocalibration and tweedie-dominance for
insurance pricing with machine learning. Insurance: Mathematics & Economics.
Denuit, M. and Robert, C. (2007). Actuariat des assurances de personnes: modélisation, tarification etprovisionnement. Economica.
Du, T., Xiong, L., Xu, C.-Y., Gippel, C. J., Guo, S., and Liu, P. (2015). Return period and risk
analysis of nonstationary low-flow series under climate change. Journal of Hydrology, 527:234–250.
Fong, C., Hazlett, C., and Imai, K. (2018). Covariate balancing propensity score for a continuous
treatment: Application to the efficacy of political advertisements. The Annals of Applied Statistics,
12(1):156–177.
Frazier, R. (2021). California’s ban on climate-informed models for wildlife insurance premiums.
Ecology L. Currents, 48:24.
Freedman, D. A. and Berk, R. A. (2008). Weighting regressions by propensity scores. Evaluation
review, 32(4):392–409.
Fuller, W. E. (1914). Flood flows. Transactions of the American Society of Civil Engineers,77(1):564–617.
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

123 / 135

References
Furht, B., Villanustre, F., Weiss, K., Khoshgoftaar, T. M., and Wang, D. (2016). Transfer learningtechniques. In Big data technologies and applications, pages 53–99. Springer.
Gneiting, T., Balabdaoui, F., and Raftery, A. E. (2007). Probabilistic forecasts, calibration andsharpness. Journal of the Royal Statistical Society: Series B (Statistical Methodology),69(2):243–268.
Goodman, L. A. (1953). Ecological regressions and behavior of individuals. American sociologicalreview, 18(6).
Goodman, L. A. (1959). Some alternatives to ecological correlation. American Journal of Sociology,64(6):610–625.
Gretton, A., Borgwardt, K. M., Rasch, M. J., Schölkopf, B., and Smola, A. (2012). A kernel
two-sample test. Journal of Machine Learning Research, 13(Mar):723–773.
Gumbel, E. J. (1941a). Probability-interpretation of the observed return-periods of floods. Eos,Transactions American Geophysical Union, 22(3):836–850.
Gumbel, E. J. (1941b). The return period of flood flows. The annals of mathematical statistics,
12(2):163–190.
Gumbel, E. J. (1958). Statistics of extremes. Columbia university press.
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

124 / 135

References
Hastie, T., Tibshirani, R., Friedman, J. H., and Friedman, J. H. (2009). The elements of statisticallearning: data mining, inference, and prediction, volume 2. Springer.
Hazen, A. (1930). Flood flows: a study of frequencies and magnitudes. Wiley.
Heckman, J. (1974). Shadow prices, market wages, and labor supply. Econometrica: journal of theeconometric society, pages 679–694.
Heckman, J. J. (1976). The common structure of statistical models of truncation, sample selection
and limited dependent variables and a simple estimator for such models. In Annals of economic andsocial measurement, volume 5, number 4, pages 475–492. NBER.
Heckman, J. J. (1979). Sample selection bias as a specification error. Econometrica: Journal of theeconometric society, pages 153–161.
Heinze-Deml, C. and Meinshausen, N. (2021). Conditional variance penalties and domain shiftrobustness. Machine Learning, 110(2):303–348.
Herbster, M. and Warmuth, M. K. (1998). Tracking the best expert. Machine learning, 32:151–178.
Hjorth, U. (1982). Model selection and forward validation. Scandinavian Journal of Statistics, pages
95–105.
Hjorth, U. (1994). Computer intensive statistical methods: Validation, model selection, and bootstrap.
Chapman and Hall/CRC.
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

125 / 135

References
Holland, P. W. (1986). Statistics and causal inference. Journal of the American statistical Association,
81(396):945–960.
Holt, D., Steel, D. G., Tranmer, M., and Wrigley, N. (1996). Aggregation and ecological effects in
geographically based data. Geographical analysis, 28(3):244–261.
Hosmer Jr, D. W., Lemeshow, S., and Sturdivant, R. X. (2013). Applied logistic regression, volume398. John Wiley & Sons.
Hume, D. (1739). A Treatise of Human Nature. Cambridge University Press Archive.
Hume, D. (1748). An Enquiry concerning Human Understanding. Cambridge University Press.
Imai, K., Olivella, S., and Rosenman, E. T. (2022). Addressing census data problems in race
imputation via fully bayesian improved surname geocoding and name supplements. Science
Advances, 8(49):eadc9824.
Imbens, G. W. and Rubin, D. B. (2015). Causal inference in statistics, social, and biomedical sciences.
Cambridge University Press.
James, G., Witten, D., Hastie, T., Tibshirani, R., et al. (2013). An introduction to statistical learning.
Springer.
Jia, L., DelSole, T., and Tippett, M. K. (2014). Can optimal projection improve dynamical model
forecasts? Journal of Climate, 27(7):2643–2655.
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

126 / 135

References
Jordan, C. (1881). Sur la serie de fourier. Camptes Rendus Hebdomadaires de l’Academie desSciences, 92:228–230.
Kantorovich, L. and Rubinstein, G. (1958). On the space of completely additive functions. Vestnic
Leningrad Univ., Ser. Mat. Mekh. i Astron., 13(7):52–59. In Russian.
Kaplan, E. L. and Meier, P. (1958). Nonparametric estimation from incomplete observations. Journalof the American statistical association, 53(282):457–481.
Karimi, H., Khan, M. F. A., Liu, H., Derr, T., and Liu, H. (2022). Enhancing individual fairness
through propensity score matching. In 2022 IEEE 9th International Conference on Data Science andAdvanced Analytics (DSAA), pages 1–10. IEEE.
Kearns, M. and Roth, A. (2019). The ethical algorithm: The science of socially aware algorithm
design. Oxford University Press.
Keren, G. (1991). Calibration and probability judgements: Conceptual and methodological issues. Acta
psychologica, 77(3):217–273.
Knowlton, R. E. (1978). Regents of the university of california v. bakke. Arkansas Law Review, 32:499.
Kolmogorov, A. (1933). Sulla determinazione empirica di una legge di distribuzione. Giornaledell’Istituto Italiano degli Attuari, 4:83–91.
Kuhn, M. and Johnson, K. (2013). Applied Predictive Modeling. Springer.
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

127 / 135

References
Kullback, S. and Leibler, R. A. (1951). On information and sufficiency. The Annals of Mathematical
Statistics, 22(1):79–86.
Kusner, M. J., Loftus, J., Russell, C., and Silva, R. (2017). Counterfactual fairness. In Advances in
Neural Information Processing Systems, pages 4066–4076.
Lee, R. D. and Carter, L. R. (1992). Modeling and forecasting us mortality. Journal of the American
statistical association, 87(419):659–671.
Li, F. and Li, F. (2019). Propensity score weighting for causal inference with multiple treatments. The
Annals of Applied Statistics, 13:2389–2415.
Li, N. and Lee, R. (2005). Coherent mortality forecasts for a group of populations: An extension of the
Lee-Carter method. Demography, 42:575–594.
Lippert-Rasmussen, K. (2020). Making sense of affirmative action. Oxford University Press.
Liu, J., Hong, Y., D’Agostino Sr, R. B., Wu, Z., Wang, W., Sun, J., Wilson, P. W., Kannel, W. B.,
and Zhao, D. (2004). Predictive value for the chinese population of the framingham chd risk
assessment tool compared with the chinese multi-provincial cohort study. Journal of the American
Medical Association, 291(21):2591–2599.
Lorenz, E. N. (1996). Predictability: A problem partly solved. In Palmer, T. and Hagedorn, R., editors,
Predictability of Weather and Climate, volume 1. Cambridge University Press.
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

128 / 135

References
Machado, A. F., Charpentier, A., Flachaire, E., Gallic, E., and Hu, F. (2024a). From uncertainty to
precision: Enhancing binary classifier performance through calibration. arXiv, 2402.07790.
Machado, A. F., Charpentier, A., Flachaire, E., Gallic, E., and Hu, F. (2024b). Probabilistic scores ofclassifiers, calibration is not enough. in progress.
Mack, T. (1991). A simple parametric model for rating automobile insurance or estimating ibnr claims
reserves. ASTIN Bulletin: The Journal of the IAA, 21(1):93–109.
Martinez, A. B. (2020). Improving normalized hurricane damages. Nature Sustainability, 3(7):517–518.
Mill, J. S. (1848). Principles of Political Economy. Baldwin, Cradock, and Joy.
Monge, G. (1781). Mémoire sur la théorie des déblais et des remblais. Histoire de l’Académie Royale
des Sciences de Paris.
Monteleoni, C. and Jaakkola, T. (2003). Online learning of non-stationary sequences. Advances in
Neural Information Processing Systems, 16.
Monteleoni, C., Schmidt, G. A., Saroha, S., and Asplund, E. (2011). Tracking climate models.
Statistical Analysis and Data Mining: The ASA Data Science Journal, 4(4):372–392.
Moodie, E. E. and Stephens, D. A. (2022). Causal inference: Critical developments, past and future.
Canadian Journal of Statistics, 50(4):1299–1320.
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

129 / 135

ReferencesMüller, A. (1997). Integral probability metrics and their generating classes of functions. Advances inapplied probability, 29(2):429–443.
Murphy, A. H. (1973). A new vector partition of the probability score. Journal of Applied Meteorology
and Climatology, 12(4):595–600.
Murray, W. and Sinnreich, R. H. (2006). The past as prologue: The importance of history to themilitary profession. Cambridge University Press.
Neddermeyer, J. C. (2009). Computationally efficient nonparametric importance sampling. Journal of
the American Statistical Association, 104(486):788–802.
Olsen, J. R., Lambert, J. H., and Haimes, Y. Y. (1998). Risk of extreme events under nonstationary
conditions. Risk Analysis, 18(4):497–510.
Owen, A. B. (2013). Monte Carlo theory, methods and examples. Stanford Lectures Notes.
Pakdaman Naeini, M., Cooper, G., and Hauskrecht, M. (2015). Obtaining well calibrated probabilities
using bayesian binning. Proceedings of the AAAI Conference on Artificial Intelligence,29(1):2901–2907.
Parthipan, R. and Wischik, D. J. (2022). Don’t waste data: Transfer learning to leverage all data formachine-learnt climate model emulation. arXiv, 2210.04001.
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

130 / 135

References
Pearl, J. and Mackenzie, D. (2018). The book of why: the new science of cause and effect. Basic
books.
Pielke, R. A. and Landsea, C. W. (1998). Normalized hurricane damages in the united states:
1925–95. Weather and forecasting, 13(3):621–631.
Pitacco, E., Denuit, M., Haberman, S., and Olivieri, A. (2009). Modelling longevity dynamics forpensions and annuity business. Oxford University Press.
Quarg, G. and Mack, T. (2004). Munich chain ladder. Blätter der DGVFM, 26(4):597–630.
Randall, D. A., Wood, R. A., Bony, S., Colman, R., Fichefet, T., Fyfe, J., Kattsov, V., Pitman, A.,
Shukla, J., Srinivasan, J., et al. (2007). Climate models and their evaluation. In Climate change
2007: The physical science basis. Contribution of Working Group I to the Fourth Assessment Reportof the IPCC (FAR), pages 589–662. Cambridge University Press.
Read, L. K. and Vogel, R. M. (2015). Reliability, return periods, and risk under nonstationarity. Water
Resources Research, 51(8):6381–6398.
Reichler, T. and Kim, J. (2008). How well do coupled models simulate today’s climate? Bulletin of theAmerican Meteorological Society, 89(3):303–312.
Reifen, C. and Toumi, R. (2009). Climate projections: Past performance no guarantee of future skill?
Geophysical Research Letters, 36(13).
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

131 / 135

References
Rizzo, M. L. and Székely, G. J. (2016). Energy distance. wiley interdisciplinary reviews: Computational
statistics, 8(1):27–38.
Robins, J. M. and Rotnitzky, A. (1992). Recovery of information and adjustment for dependent
censoring using surrogate markers. In AIDS epidemiology: methodological issues, pages 297–331.
Springer.
Rojas-Carulla, M., Schölkopf, B., Turner, R., and Peters, J. (2018). Invariant models for causal
transfer learning. Journal of Machine Learning Research, 19(36):1–34.
Rosenbaum, P. R. and Rubin, D. B. (1983). The central role of the propensity score in observational
studies for causal effects. Biometrika, 70(1):41–55.
Rubin, D. B. (1973). Matching to remove bias in observational studies. Biometrics, pages 159–183.
Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized
studies. Journal of educational Psychology, 66(5):688.
Rudin, W. (1966). Real and Complex Analysis. McGraw-hill New York.
Sabbagh, D. (2007). Equality and transparency: A strategic perspective on affirmative action in
American law. Springer.
Salas, J. D. and Obeysekera, J. (2014). Revisiting the concepts of return period and risk for
nonstationary hydrologic extreme events. Journal of hydrologic engineering, 19(3):554–568.
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

132 / 135

References
Sartin, E. B., Metzger, K. B., Pfeiffer, M. R., Myers, R. K., and Curry, A. E. (2021). Facilitating
research on racial and ethnic disparities and inequities in transportation: Application and evaluationof the bayesian improved surname geocoding (bisg) algorithm. Traffic injury prevention,
22(sup1):S32–S37.
Satten, G. A. and Datta, S. (2001). The kaplan–meier estimator as an inverse-probability-of-censoring
weighted average. The American Statistician, 55(3):207–210.
Schmidt, G. (2024). Climate models can’t explain 2023’s huge heat anomaly — we could be inuncharted territory. Nature, 627:467.
Sedgwick, P. (2015). Understanding the ecological fallacy. British Medical Journal, 351.
Shakespeare, W. (1610). The Tempest. Oxford Paperbacks.
Shimodaira, H. (2000). Improving predictive inference under covariate shift by weighting the
log-likelihood function. Journal of statistical planning and inference, 90(2):227–244.
Silver, N. (2012). The signal and the noise: Why so many predictions fail-but some don’t. Penguin.
Smirnov, N. (1948). Table for estimating the goodness of fit of empirical distributions. The annals ofmathematical statistics, 19(2):279–281.
Sorbero, M. E., Euller, R., Kofner, A., and Elliott, M. N. (2022). Imputation of race and ethnicity inhealth insurance marketplace enrollment data, 2015-2022 open enrollment periods. RAND.
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

133 / 135

References
Stone, M. (1974). Cross-validatory choice and assessment of statistical predictions. Journal of theroyal statistical society: Series B (Methodological), 36(2):111–133.
Székely, G. J. (2003). E-statistics: The energy of statistical samples. Bowling Green State University,
Department of Mathematics and Statistics Technical Report, 3(05):1–18.
Task Committee on Hydrology Handbook of Management Group D of ASCE (1996). Hydrology
handbook. American Society of Civil Engineers.
Taylor, G. C. (1977). Separation of inflation and other effects from the distribution of non-life
insurance claim delays. ASTIN Bulletin: The Journal of the IAA, 9(1-2):219–230.
Turner, R. (2015). The way to stop discrimination on the basis of race. Stanford Journal of Civil
Rights & Civil Liberties, 11:45.
Twain, M. and Warner, C. D. (1874). The Gilded Age: a tale of today. Penguin.
Van Calster, B., McLernon, D. J., Van Smeden, M., Wynants, L., and Steyerberg, E. W. (2019).
Calibration: the achilles heel of predictive analytics. BMC medicine, 17(1):1–7.
Van der Vaart, A. W. (2000). Asymptotic statistics. Cambridge university press.
Vapnik, V. N. and Chervonenkis, A. Y. (1971). On uniform convergence of the frequencies of events totheir probabilities. Teoriya Veroyatnostei i ee Primeneniya, 16(2):264–279.
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

134 / 135

References
Verrall, R. (1996). Claims reserving and generalised additive models. Insurance: Mathematics andEconomics, 19(1):31–43.
Voicu, I. (2018). Using first name information to improve race and ethnicity classification. Statistics
and Public Policy, 5(1):1–13.
von Mises, R. (1928). Wahrscheinlichkeit Statistik und Wahrheit. Springer.
von Mises, R. (1939). Probability, statistics and truth. Macmillan.
Wager, S. and Athey, S. (2018). Estimation and inference of heterogeneous treatment effects using
random forests. Journal of the American Statistical Association, 113(523):1228–1242.
Wasserstein, L. N. (1969). Markov processes over denumerable products of spaces, describing large
systems of automata. Problemy Peredachi Informatsii, 5(3):64–72.
Weinkle, J., Landsea, C., Collins, D., Musulin, R., Crompton, R. P., Klotzbach, P. J., and Pielke Jr, R.
(2018). Normalized hurricane damage in the continental united states 1900–2017. Naturesustainability, 1(12):808–813.
Wilson, P. W., Castelli, W. P., and Kannel, W. B. (1987). Coronary risk prediction in adults (theframingham heart study). The American journal of cardiology, 59(14):G91–G94.
Wilson, P. W., D’Agostino, R. B., Levy, D., Belanger, A. M., Silbershatz, H., and Kannel, W. B.
(1998). Prediction of coronary heart disease using risk factor categories. Circulation,
97(18):1837–1847.
 @freakonometrics § freakonometrics

freakonometrics.hypotheses.org – Arthur Charpentier, 2024 (KU Leuven)

135 / 135