<figure><img src="image_1.png" /></figure>

HAL
open science

# Estimation of extreme risk measures with neural networks

Stéphane Girard, Michaël Allouche, Emmanuel Gobet

► To cite this version:

Stéphane Girard, Michaël Allouche, Emmanuel Gobet. Estimation of extreme risk measures with neural networks. JDS2025 - 56èmes Journées de Statistique de la SfdS, Jun 2025, Marseille, France. pp.1-6. hal-05101894

HAL Id: hal-05101894

https://hal.science/hal-05101894v1

Submitted on 6 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# ESTIMATION OF EXTREME RISK MEASURES WITH NEURAL NETWORKS

Michaël Allouche¹, Stéphane Girard² & Emmanuel Gobet³

¹ Kaiko - Quantitative Data, 2 rue de Choiseul, 75002 Paris, France;

² Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, 38000 Grenoble, France;
stephane.girard@inria.fr.

³ Centre de Mathématiques Appliquées (CMAP), Ecole Polytechnique and CNRS,
Institut Polytechnique de Paris, Route de Saclay, 91128 Palaiseau Cedex, France;
emmanuel.gobet@polytechnique.edu.

**Résumé.** Nous proposons de nouvelles paramétrisations de réseaux de neurones afin d'estimer des mesures de risque extrêmes tels que les moments conditionnels de queue, dans des contextes de lois à queues lourdes. L'estimateur proposé est capable d'extrapoler dans les queues de distribution grâce à une extension de la condition de second-ordre à un ordre arbitraire. Le taux de convergence de l'erreur uniforme entre le log-moment conditionnel de queue et son approximation par le réseau de neurones est établi. La performance en pratique de l'estimateur obtenu est comparée sur des données simulées à celle de concurrents issus de la théorie des valeurs extrêmes et incluant une réduction du biais. Il est démontré que notre méthode les surpasse dans les situations difficiles où les autres estimateurs échouent presque tous. Enfin, l'estimateur par réseaux de neurones est testé sur des données réelles afin d'étudier le comportement de rendements extrêmes de crypto-monnaies.

**Mots-clés.** Statistique des valeurs extrêmes, mesure de risque, lois à queue lourde, réseau de neurones.

**Abstract.** We propose new parameterizations for neural networks in order to estimate extreme risk measures, such as conditional tail moments, in heavy-tailed settings. The proposed neural network estimator is able to extrapolate in the distribution tails thanks to an extension of the usual extreme-value second-order condition to an arbitrary order. The convergence rate of the uniform error between the log-conditional tail moment and its neural network approximation is established. The finite sample performance of the neural network estimator is compared to bias-reduced extreme-value competitors on simulated data. It is shown that our method outperforms them in difficult heavy-tailed situations where other estimators almost all fail. Finally, the neural network estimator is tested on real data to investigate the behavior of cryptocurrency extreme loss returns.

**Keywords.** Extreme value theory, risk measure, heavy-tailed distribution, neural networks.
# 1 Introduction

The Value at Risk (VaR) is the quantile at a given confidence level $(1 - \alpha)$:

$$ \mathrm{VaR}(1 - \alpha) = q(1 - \alpha) = \inf\{x \in \mathbb{R}, F(x) \ge 1 - \alpha\}, $$

where $\alpha \in (0, 1)$ and with $F$ the cumulative distribution function of the random variable $X$ of interest ($X$ is usually the opposite of a loss and $\alpha$ is close to 0). Besides, the ES is defined as the average of the quantile function above the confidence level $(1 - \alpha)$:

$$ \mathrm{ES}(1 - \alpha) = \frac{1}{\alpha} \int_{0}^{\alpha} q(1 - u) \, du. $$

When the distribution of $X$ is continuou, $\mathrm{ES}(1 - \alpha)$ coincides with the conditional expectation of $X$ given that it exceeds the VaR $q(1 - \alpha)$:

$$ \mathrm{ES}(1 - \alpha) = \mathbb{E}(X | X > q(1 - \alpha)), $$

and it is referred to as Conditional Tail Expectation (CTE). In this work, we consider more generally Conditional Tail Moments (CTMs) of order $p > 0$, defined in [5] as

$$ \mathrm{CTM}_p(1 - \alpha) = \mathbb{E}(X^p | X > q(1 - \alpha)) = \frac{1}{\alpha} \int_0^\alpha q^p(1 - u) \, du \quad (1) $$

under the assumption of a continuous distribution. Clearly, $\mathrm{CTM}_1 = \mathrm{ES}$. Nonparametric estimators have been introduced in [3, 4] based on the empirical counterpart of (1):

$$ \widehat{\mathrm{CTM}}_p(1 - \alpha) = \frac{1}{[n\alpha]} \sum_{j=1}^{[n\alpha]} X_{n-j+1,n}^p \quad \text{with} \quad \widehat{\mathrm{ES}}(1 - \alpha) = \widehat{\mathrm{CTM}}_1(1 - \alpha), \quad (2) $$

and where $X_{1,n} \le \dots \le X_{n,n}$ denote the order statistics associated with the initial $n$-sample $\{X_1, \dots, X_n\}$. We consider extreme risks, i.e. the situation where $\alpha$ is so small that the VaR is larger than the maximal observation: $q(1 - \alpha) > X_{n,n}$. This is asymptotically the case when $n\alpha$ is small, and in this challenging situation, the naive estimator (2) cannot be used to estimate the extreme ES or CTM. To overcome this problem, we propose an extrapolation formula dedicated to heavy-tailed distributions and leveraging extreme-value theory.

# 2 An approximation result

Let $\{X_1, \dots, X_n\}$ be an independent sample from an unknown cumulative distribution function $F$ assumed to be continuous. The associated order statistics are denoted by $X_{1,n} \le \dots \le X_{n,n}$. We address the estimation of $\mathrm{ES}(1 - \alpha_n)$ at an extreme level $\alpha_n$ i.e. such that $n\alpha_n \to 0$ as $n \to \infty$. We moreover focus on heavy-tailed distributions whose survival function $(1 - F)$ is regularly-varying with index $-1/\gamma$, where $\gamma > 0$ is called the tail-index. Our main assumption can be written as
(A) $(1-F) \in \mathcal{RV}_{-1/\gamma}$ with $\gamma > 0$ i.e.

$$ \lim_{t \to \infty} \frac{1-F(tx)}{1-F(t)} = x^{-1/\gamma}, \text{ for all } x > 0. $$

The index $\gamma$ tunes the tail heaviness of $F$: the larger the index, the heavier the right tail. Examples include the (generalized) Pareto, Burr, Fisher, Inverse gamma and Student distributions. Equivalently, the tail quantile function $t > 1 \mapsto U(t) = q(1 - 1/t)$ can be rewritten as $U(t) = t^{\gamma}L(t)$ where $L \in \mathcal{RV}_0$ is called a slowly-varying function. It is easily shown that, if (A) holds with $p\gamma \in (0, 1)$, then $\text{CTM}_p(1 - \alpha)$ is finite for any $\alpha \in (0, 1)$. We also consider a $J$-th order condition ($J \ge 2$), introduced in [10], on the slowly-varying function to control the rate of convergence in (A):

(B$_J$) There exist for all $j \in \{2, \dots, J\}$, functions $A_j$ with $A_j(t) \to 0$ as $t \to \infty$ and an asymptotically constant sign, $|A_j| \in \mathcal{RV}_{\rho_j}$ and $\rho_j \le 0$ such that

$$ \log L(tz) - \log L(t) = \sum_{j=2}^{J} \prod_{\ell=2}^{j} A_{\ell}(t) R_{j}(z) + o\left(\prod_{j=2}^{J} A_{j}(t)\right), $$

as $t \to \infty$ for all $z > 0$, where:

$$ R_j(z) := \int_1^z z^{\rho_2-1} \int_1^{z_2} z^{\rho_3-1} \dots \int_1^{z_{j-1}} z^{\rho_j-1} dz_j \dots dz_3 dz_2. $$

This condition refines the initial heavy-tail assumption and is a classical device in extreme-value analysis for extrapolating beyond the observation range. For instance, when $J = 2$, we recover the classical second-order condition [8, Equation (13)]. Moreover, $J = 3$ and $J = 4$ yield back respectively the third-order [6] and fourth-order conditions [7]. For most of heavy-tailed distributions, $A_1, \dots, A_J$ are power functions, which will be our assumption in the sequel:

(C$_J$) $A_j(t) = c_j t^{\rho_j}$ where $c_j \ne 0$ and $\rho_j < 0$ for $j \in \{2, \dots, J\}$.

When $J = 2$, (C$_J$) amounts to supposing that the underlying distribution belongs to the Hall-Welsh class [9]. Finally, we shall also need the following assumption.

(D) $L$ is differentiable and $|(\log L)'| \in \mathcal{RV}_{\rho_2-1}$ with $\rho_2 < 0$.

Assume (A) holds with $p\gamma \in (0, 1)$, and consider the slowly-varying function $t \mapsto L(t) = t^{-\gamma}U(t)$. The following two functions are then introduced:

$$ (y_1, y_2) \in [0, \infty)^2 \mapsto \varphi(y_1, y_2) = \log L(\exp(y_1 + y_2)) - \log L(\exp(y_2)), $$

$$ u \in (0, 1] \mapsto \Psi_p(u) = \log \left( \int_0^1 w^{-p\gamma} \frac{L^p(1/(wu))}{L^p(1/u)} dw \right). $$
The proposed approach consists in estimating these functions $\varphi$ and $\Psi_p$ using neural networks (NNs). In [1], we have established in a similar statistical framework that the log-spacings between two quantiles can be written with $\varphi$ as

$$ \log q(1 - \alpha) - \log q(1 - \delta) = \gamma \log(\delta/\alpha) + \varphi(\log(\delta/\alpha), \log(1/\delta)). $$

The log-spacings associated with the CTM can be rewritten thanks to $\varphi$ and $\Psi_p$.

**Proposition 1** If (A) holds with $p\gamma \in (0, 1)$, then, for all $(\alpha, \delta) \in (0, 1)^2$:

$$ \begin{aligned} \log \text{CTM}_p(1 - \alpha) - \log \text{CTM}_p(1 - \delta) &= p\gamma \log(\delta/\alpha) + p\varphi(\log(\delta/\alpha), \log(1/\delta)) \\ & \quad + \Psi_p(\alpha) - \Psi_p(\delta) \\ &:= g_p(\alpha, \delta). \end{aligned} $$

Proposition 1 allows us to derive an extrapolation principle for the CTM. Indeed, the CTM at an extreme level $1 - \alpha_n$ (such that $n\alpha_n \to 0$ as $n \to \infty$) can be deduced from the CTM at an intermediate level $1 - \delta_n$ (such that $n\delta_n \to \infty$ as $n \to \infty$) since

$$ \text{CTM}_p(1-\alpha_n) = \text{CTM}_p(1-\delta_n)(\delta_n/\alpha_n)^{p\gamma} \exp \left( p\varphi \left( \log(\delta_n/\alpha_n), \log(1/\delta_n) \right) + \Psi_p(\alpha_n) - \Psi_p(\delta_n) \right), $$
or equivalently

$$ \text{CTM}_p(1 - \alpha_n) = \text{CTM}_p(1 - \delta_n) \exp(g_p(\alpha_n, \delta_n)). $$

This kind of extrapolation method is widely used in extreme-value theory, see for instance Weissman estimator [11] dedicated to extreme quantiles. The challenge is to design an extrapolation function $g_p$ in a way that the approximation of $\text{CTM}_p(1 - \alpha_n)$ is as stable as possible with respect to the anchor level $\delta_n$. The implementation of this idea requires the estimation of two real quantities, namely $\text{CTM}_p(1 - \delta_n)$ and $\gamma$ as well as the two functions $\varphi$ and $\Psi_p$. In the following, we focus on a nonparametric estimation of $\varphi$ and $\Psi_p$ using NNs whose first step consists in approximating the two unknown functions.

**Theorem 2** Let $L$ be a slowly-varying function verifying $(\mathbf{B}_J)$, $(\mathbf{C}_J)$ with $J \ge 2$ and $(\mathbf{D})$. Let $p\gamma \in (0, 1)$ and $\varepsilon \in (0, |\rho_2|)$. Then, there exists a NN approximation of the CTM involving $K_1 \ge 6$ ReLU units and $K_2 = J(J-1)/2$ eLU units such that

$$ \frac{1}{p} \inf_{\theta \in \Theta} \left| \log \text{CTM}_p(1 - \alpha_n) - \log \widetilde{\text{CTM}}_{p,\theta}^K (1 - \alpha_n; 1 - \delta_n) \right| = O(K_1^{-1\vee(\rho_2+\varepsilon)}) + O((\delta_n/\alpha_n)^\varepsilon \alpha_n^{-\bar{\rho}_J}), $$
where $(\alpha_n)$ and $(\delta_n)$ are probability sequences such as $\delta_n \to 0$ and $\alpha_n/\delta_n \to 0$ as $n \to \infty$.

The approximation rate does not depend on the tail heaviness $\gamma$, but rather on the parameters $\rho_2$ and $\bar{\rho}_J$ stemming from the $J$-th order condition. Since $\bar{\rho}_J$ is a decreasing function of $J$, choosing $J > 2$ may be of interest to reduce the approximation error while usual extreme-value estimators rely on first- or second-order approximations. The approximation rate is also driven by the complexity of the NN which can be quantified by both $K_1$ the number of ReLU units and $K_2$ the number of eLU units via $J$. Finally, note that the approximation error is an increasing function of $\alpha_n$: The further in the tail the risk measure is, the better its approximation.
### 3 Application to the estimation of the CTM

Here, we let $\delta_n = k/n$ where $k = k_n$ is an intermediate sequence i.e. such that $k \to \infty$ and $k/n \to 0$ as $n \to \infty$. The CTM at the intermediate level $\delta_n$ is estimated by its empirical counterpart (2) and the CTM at the extreme level $\alpha_n$ is thus estimated by

$$ \widehat{\text{CTM}}_{p,\theta}^{\text{NN}}(1-\alpha_n; 1-k/n) := \left(\frac{1}{k}\sum_{j=1}^{k} X_{n-j+1,n}^{p}\right) (k/(n\alpha_n))^{p\hat{\theta}_0} \exp\left(p\hat{\varphi}_{\hat{\theta}_2}^{K_2}(\log(k/(n\alpha_n)), \log(n/k))\right) \\ \times \exp\left(\tilde{\Psi}_{p,\hat{\theta}_1}^{K_1}(\alpha_n) - \tilde{\Psi}_{p,\hat{\theta}_1}^{K_1}(k/n)\right) \\ = \left(\frac{1}{k}\sum_{j=1}^{k} X_{n-j+1,n}^{p}\right) \exp\left(\tilde{g}_{p,\hat{\theta}}^{K}(\alpha_n, k/n)\right), $$

where $\hat{\theta} = (\hat{\theta}_0, \hat{\theta}_1, \hat{\theta}_2)$ is computed by fitting the NN model of the log-spacings to the empirical ones. More specifically, Proposition 1 shows that, for all $k \in \{1, \dots, n\}$ and $i \in \{k+1, \dots, n\}$, one has $S_{i,k} := \log \text{CTM}_p(1-i/n) - \log \text{CTM}_p(1-k/n) = g_p(i/n, k/n)$. The empirical estimator is deduced from (2) and given by

$$ \hat{S}_{i,k} := \log \left( \frac{1}{i} \sum_{j=1}^{i} X_{n-j+1,n}^{p} \right) - \log \left( \frac{1}{k} \sum_{j=1}^{k} X_{n-j+1,n}^{p} \right), $$

while the NN model is $g_{J\phi}(i/n, k/n)$. In practice, small indices $i$ and $k$ are discarded since the associated empirical estimates of the CTM suffer from a large variance. To this end, we consider $\zeta \in [0, 1)$ and minimize the following distance between the above two estimations of the $N = (n - 2 - [\zeta n])(n - 1 + [\zeta n])/2$ log-spacings:

$$ \hat{\theta} = \arg\min_{\theta \in \Theta} \frac{1}{N} \sum_{k=2+[\zeta n]}^{n-1} \sum_{i=1+[\zeta n]}^{k-1} \left| \hat{S}_{i,k} - g_{J\phi}(i/n, k/n) \right|. $$

The finite sample behavior of the estimator is illustrated in [2] both on simulated and real data sets. An overview of the results will be presented during the talk.

### References

[1] M. Allouche, S. Girard, and E. Gobet. Estimation of extreme quantiles from heavy-tailed distributions with neural networks. *Statistics and Computing*, 34:12, 2024.

[2] M. Allouche, S. Girard, and E. Gobet. Learning extreme expected shortfall and conditional tail moments with neural networks. application to cryptocurrency data. *Neural Networks*, 182:106903, 2025.
[3] V. Brazauskas, B. Jones, M. Puri, and R. Zitikis. Estimating conditional tail expectation with actuarial applications in view. *Journal of Statistical Planning and Inference*, 138(11):3590–3604, 2008.

[4] E. Deme, S. Girard, and A. Guillou. Reduced-bias estimators of the conditional tail expectation for heavy-tailed distributions. In M. Hallin et al., editor, *Mathematical Statistics and Limit Theorems*, pages 105–123. Springer, 2015.

[5] J. El Methni, L. Gardes, and S. Girard. Non-parametric estimation of extreme risk measures from conditional heavy-tailed distributions. *Scandinavian Journal of Statistics*, 41(4):988–1012, 2014.

[6] I. Fraga Alves, L. de Haan, and T. Lin. Third order extended regular variation. *Publications de l'Institut Mathématique*, 80(94):109-120, 2006.

[7] Y. Goegebeur and T. de Wet. Estimation of the third-order parameter in extreme value statistics. *Test*, 21(2):330-354, 2012.

[8] M. I. Gomes, L. de Haan, and L. Peng. Semi-parametric estimation of the second order parameter in statistics of extremes. *Extremes*, 5(4):387–414, 2002.

[9] P. Hall and A. W. Welsh. Adaptive estimates of parameters of regular variation. *The Annals of Statistics*, 13:331–341, 1985.

[10] X. Q. Wang and S. H. Cheng. General regular variation of the n-th order and 2nd order Edgeworth expansions of the extreme value distribution. II. *Acta Mathematica Sinica (English Series)*, 22(1):27-40, 2006.

[11] I. Weissman. Estimation of parameters and large quantiles based on the k largest observations. *Journal of the American Statistical Association*, 73(364):812–815, 1978.