# Why a better goodness-of-fit test? A modified Berk-Jones statistical test and application to climate data

Philippe Ear, Elena Di Bernardino, Thomas Laloë, Adrien Lambert, Magali Troin

<figure><img src="image_1.png" /></figure>

## To cite this version:

Philippe Ear, Elena Di Bernardino, Thomas Laloë, Adrien Lambert, Magali Troin. Why a better goodness-of-fit test? A modified Berk-Jones statistical test and application to climate data. International Conference on Statistics and Data Science 2024 (ICSDS24), Dec 2024, Nice, France. 2024. [hal-05010492](https://hal.science/hal-05010492v1)

HAL Id: hal-05010492

https://hal.science/hal-05010492v1

Submitted on 28 Mar 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A modified Berk-Jones statistical test and application to climate data

<figure><img src="image_3.png" /></figure>

## A tail deviation's sensitive goodness-of-fit test for daily precipitation modelling

Philippe Ear* (1,2), Elena Di Bernardino (1), Thomas Laloë (1), Adrien Lambert (2), Magali Troin (2)
(1) Université Côte d'Azur, France (2) Hydroclimat, France

<figure><img src="image_4.png" /></figure>

*corresponding author : philippe.ear@hydroclimat.com

## Why a better goodness-of-fit test ?

Classical goodness-of-fit (GOF) tests such as :

* The Kolmogorov-Smirnov test

* The Cramer-Von Mises test

* The Anderson-Darling test
are well-known and widely used.

But they lack power in tails, especially for heavy-tailed distributions.

The main reason is that their statistics are based on the amplitude between the theoretical CDF and observed CDF.

For deviation in tails, especially for heavy-tailed distributions, very small deviations can result in large quantile differences.

## The Berk-Jones statistical test

The **Berk-Jones statistic** is based on the significance of the deviation between the theoretical and observed CDF.

Let $x = (x_{(1)}, ..., x_{(n)})$ be samples from the random variable $X$ and let $F$ be the CDF of $X$ under $H_0$.

Let $u_{(i)} = F(x_{(i)})$, the ordered statistics.

A well known result is that each $u_{(i)}$ are random samples from $U_{(i)} \sim \text{Beta}(i, n - i + 1)$. Let's now define

$$p_{(i)} := P(U_{(i)} < u_{(i)})$$

The statistic $M_n$ is then defined by :

$$M_n^+ = \min(p_{(i)}), M_n^- = \min(1 - p_{(i)})$$

$$M_n = \min\{M_n^+, M_n^-\}$$

The p-value can then be computed using some algorithms.

The Berk-Jones GOF test shows **more power** than other classics GOF tests for a given significance.

Theroretical proofs and algorithms for p-value computation can be found in (Moscovich et al, 2016)

## Penalized and modified test

The Berk-Jones GOF :

* Can lack power in distribution tails

* Gives indications on where the deviations are

We introduce two variation of the Berk-Jones GOF test :

* **Penalized Berk-Jones test (PBJ)**

* **Left (or right) Berk-Jones test (LBJ and RBJ)**

The **Penalized Berk-Jones test** penalizes the values $p_{(i)}$ based on the **absolute error** between the **theoretical and empirical quantile** of the given rank.

$$k_i = \frac{p_{(i)}}{\max\left(1, \left|F^{-1}\left(\frac{i-1}{n}\right), x_{(i)}\right|\right)}$$

The **Left Berk-Jones test** uses the index of the rejected $p_{(i)}$ to find the smallest rank from which the distribution fits the data. See (Ear et al, 2024, under review) for more details.

## Remarks:

The PBJ works best when the considered data is scaled such that high errors are typically in the low hundreds, or high tens for the penalization to be effective without being overly strict.

The **RBJ** is basically the LBJ but with the reversed data, using the **PBJ** rejections index and $k_i$.

## Empirical power and significance level assessment

For all BJ related tests, the first 1% of $p_{(i)}$ were ignored because of very high sensitivity on this end of the tail and all $H_0$ distributions parameters were estimated using MLE. The experiment was repeated 1000 times for each sample size.

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

* **Similar significance ($\alpha$ = 5%**) of BJ, PBJ and RBJ compared to benchmark

* **Higher power** for sample size >100 across all scenarios compared to benchmark

* **Faster convergence of power** in case of a stitch distribution for BJ and PBJ

* **Ability to detect divergence point** with **high accuracy** when comparing two distributions

## Application to daily precipitation modelling

<figure><img src="image_11.png" /><figcaption>Modelisation of ERA5L and CERRAL DJF daily precipitation data from 1985-2010</figcaption></figure>

<figure><img src="image_12.png" /></figure>

<figure><img src="image_13.png" /></figure>

<figure><img src="image_14.png" /></figure>

This method allows to mitigate a lot of potentially **extreme fitting error** at the cost of more parameters to estimate for the other used distributions.

In our application, we end up with semi-parametric and parametric distributions that can be then used for bias correction with improvements especially in terms of extreme errors reduction, compared to other classical methods.

## Conclusion

We introduced variations of the BJ test:

* The PBJ which is a penalized version of the BJ test

* The LBJ and RBJ which can give information on where the deviation from $H_0$ starts

Using simulations, we found reasonable to conclude that the PBJ is able to improve the power of the original test while preserving its significance, while the RBJ and LBJ allowed for a precise distribution's deviation detection.