# Self-Adaptive Quantiles for Precipitation Forecasting

Zied Ben Bouallègue, Maxime Taillardat

► To cite this version:

Zied Ben Bouallègue, Maxime Taillardat. Self-Adaptive Quantiles for Precipitation Forecasting. Tellus A: Dynamic meteorology and oceanography, 2025, 77 (1), pp.160-172. 10.16993/tellusa.4110. hal-05114258

HAL Id: hal-05114258

https://hal.science/hal-05114258v1

Submitted on 16 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

# Self-Adaptive Quantiles for Precipitation Forecasting

ORIGINAL RESEARCH
PAPER

ZIED BEN BOUALLÈGUE ID

MAXIME TAILLARDAT ID

*Author affiliations can be found in the back matter of this article

<figure><img src="image_3.png" /></figure>

STOCKHOLM
UNIVERSITY PRESS

## ABSTRACT

How much rain can we expect in Toulouse on Wednesday next week? It is impossible to provide a precise and definitive answer to this question due to the limited predictability of the atmosphere. Ideally, a forecast would be probabilistic, for example expressed in the form of a probability of, say, having at least some rain. However, for some forecast users and applications, an answer expressed in millimeter of rain per 24 h would be needed. A so-called *point-forecast* can be the output of a single deterministic model. But with ensemble forecasts at hand, how to summarize optimally the ensemble information into a single outcome? The ensemble mean or quantile forecasts are commonly used and proved useful in certain circumstances. In this study, we suggest a new type of *point-forecasts*, the crossing-point quantile, and argue that it could be better suited for precipitation forecasting than existing approaches, at least for some users. More precisely, for a well-calibrated predictive distribution, the crossing-point quantile is the optimal forecast in terms of Peirce skill score (and equivalently in terms of area under the relative operating characteristic curve) for any event of interest. Along a theoretical proof, we present an application to daily precipitation forecasting over France and discuss the necessary conditions for optimality.

## CORRESPONDING AUTHOR:

Zied Ben Bouallègue

The European Centre for
Medium-Range Weather
Forecasts, Reading, UK

zied.benbouallegue@ecmwf.int

## KEYWORDS:

optimal forecast; ensemble
forecasting; elicitation;
crossing-point

## TO CITE THIS ARTICLE:

Ben Bouallègue, Z. and
Taillardat, M. (2025)
Self-Adaptive Quantiles for
Precipitation Forecasting.
*Tellus A: Dynamic Meteorology
and Oceanography* 77(1):
160-172. DOI: https://doi.org/
10.16993/tellusa.4110
# 1 INTRODUCTION

While weather forecasting was approached as a deterministic initial value problem at the beginning of the 20th century (Bjerknes, 1904), ensemble prediction systems are nowadays common practice in meteorology and many other scientific fields (Chen, 2024). Ensembles consist of a set of plausible weather outcomes that allow the uncertainty about the future state of the atmosphere to be estimated (Leith, 1974). Accounting for forecast uncertainty in decision-making has proven more valuable than relying on a single best-guess forecast (Cooke, 1906; Roulston and Smith, 2004, and others in between).

Forecast end-users' ability to deal with uncertainty is, however, heterogeneous. For some end-users, the uncertainty associated with a forecast is useful (Joslyn and Savelli, 2010; Morss et al., 2008), while for others, this information is difficult to apprehend (Fundel et al., 2019). Strategies to account for forecast uncertainty can consist in extracting a summary forecast that would, in the long term, optimize one's decision-making process.

Moreover, automation of forecasting tasks are developed by meteorological agencies to mitigate production overload and limited resources (Pagano et al., 2016). By nature, forecast automation relies on maximizing a measure of accuracy. In this context, the question arises of how the diversity of information generated by an ensemble, on the one hand, and the automation requirement, on the other hand, can fit together while acknowledging the diversity of users' needs. One approach is to derive a single actionable forecast potentially suitable for a wide range of users.

So-called best-guess forecasts can be derived from an ensemble by using, for example, clustering. Different clustering techniques exist to create consistent scenarios, with one single model representing a cluster

(Ono, 2023). However, the use of a single model is questionable: the definition of the best member is a highly multidimensional task (Roulston and Smith, 2003), and the concept of the dominance of one model among all realizations is sometimes considered a flawed concept (Bright and Nutter, 2004).

Alternatively, the whole ensemble can be used to create a pseudo-deterministic meteorological field, the so-called consensus forecast (Roebber, 2010). For example, optimal forecast with respect to square and/or absolute errors can be derived using genetic programming (Bakhshaii and Stull, 2009) or spatially patched precipitation field from ensemble members closest to the mean (Schwartz et al., 2014). More recently, optimal threshold optimization based on F2 and Equitable Threat Score has been empirically investigated by Bouttier and Marchal (2024). A common aspect of all these studies on consensus forecasts is that optimization relies on the definition of a measure of accuracy.

What is the ideal consensus forecast for a given measure of accuracy? The answer to this question is related to the concept of forecast *elicitation* or *consistency with a scoring rule* (Ziegel, 2016). The development of a mathematical framework for a formal definition of elicitation is out of scope here, but we introduce this concept based on two simple examples: (1) the ensemble mean is *consistent* with the root mean squared error (RMSE) and (2) quantile forecasts are *consistent* with quantile scores (QS). A special case of the latter is the 50% quantile (or median), which is consistent with the mean absolute error.

**Figure 1** illustrates the concept of *elicitation* with two examples. While more details about the data and the different forecast types are provided below, we can make the following remarks. The ensemble mean (*em*) is the best-performing forecast in terms of RMSE for all lead

<figure><img src="image_4.png" /><figcaption>**Figure 1** The best forecast as measured by a given performance measure can be the worst forecast with another performance metric. Verification results of precipitation point-forecasts in terms of (a) root mean squared error (RMSE) and (b) quantile score for a probability level of 98% (QSmax) as a function of the forecast lead time. For both performance metrics, the lower the better. The different types of forecasts are discussed in the text.</figcaption></figure>
times in Figure 1(a). The concept of elicitation is powerful because one does not need to test if the ensemble mean is the best forecast in terms of RMSE for a given application, but rather should know beforehand that it is the case. Similarly, the quantile forecast (mx) is the best forecast in terms of the quantile score QSmax for all lead times in Figure 1(b), as expected. In this example, em is one of the worst performing forecasts in terms of QSmax, and mx is the worst performing forecast in terms of RMSE. In other words, the best forecast for one application can be the worst for the other one.

In this manuscript, we introduce a new type of point-forecast, the crossing-point quantile, and demonstrate that this forecast is optimal in terms of Peirce skill score (PSS, Peirce, 1884) in case of a reliable ensemble system, whatever the event of interest. The work presented here is based on the crossing-point forecast (CPF) developed and explored in previous studies. Ben Bouallègue (2021) showed that the CPF is consistent with the Diagonal score, a threshold/quantile-weighted continuous ranked probability score introduced in Ben Bouallègue et al. (2018).

This paper is organized as follows: Section 2 introduces the point-forecasts used for comparison together with a description of the theoretical properties of the crossing-point quantile. Section 3 displays illustrations of forecasts and observations alongside an analysis of the consistency between them. Section 4 discusses verification results before concluding in Section 5.

## 2 FROM ENSEMBLE TO POINT-FORECASTS

### 2.1 BENCHMARKING

We start with the ensemble prediction system run at the European Centre for Medium-Range Weather Forecasts

(ECMWF). The control member and the 50 perturbed ensemble members serve as a basis for deriving point-forecasts. A point-forecast is defined as a single-valued forecast expressed in the unit of the weather variable of interest. A deterministic forecast for temperature at, say, Reading, UK, tomorrow is a point-forecast. Another example is the ensemble mean for the same quantity.

In this study, we propose to compare the performance and characteristics of different point-forecasts with a focus on daily precipitation and time ranges up to the medium range. As a benchmark, we consider the following point-forecasts:

* a deterministic forecast, here the ensemble control member (ctrl),

* the ensemble mean (em), which is typically a smoothed field at longer time ranges as predictability decreases,

* the maximum of all ensemble members (mx), which can also be interpreted as a quantile forecast at probability level $\frac{M}{M+1}$ with M the ensemble size¹,

* the quantile at a probability level of 70% conditioned on that at least half of the ensemble members indicates rain ($q_{70}^M$). This latter is the consensus forecast used at Météo-France as a medium-range precipitation forecast.

These four types of point-forecasts are illustrated with a synthetic example in Figure 2(a).

### 2.2 SELF-ADAPTIVE QUANTILES

Let us now remind the reader of the definition of the crossing-point forecast. For this, consider a condition called single intersection condition. Denoting as F and G as two cumulative distribution functions, F and G satisfy

<figure><img src="image_5.png" /><figcaption>**Figure 2** Types of point-forecasts investigated in this study as illustrated with a synthetic example: (a) a deterministic forecast (ctrl), a forecast ensemble distribution (solid gray line) with the corresponding ensemble mean (em), ensemble maximum (mx) and conditional 70% quantile ($q_{70}^M$); (b) the model climate (dashed gray line), the intersection point with the ensemble distribution, and the corresponding self-adaptive quantile ($q_{ctrl}^M$); and (c) same as (b) but when considering the observation climate and a calibrated ensemble (dark gray) to define the self-adaptive quantile ($q_{ctrl}^O$).</figcaption></figure>
the single intersection condition if there is one unique q such that:

$$x \ge q \Rightarrow F(x) \ge G(x) \qquad (1)$$

and

$$x < q \Rightarrow F(x) < G(x). \qquad (2)$$

In this situation, the intersection between F and G is defined by the point (q, F(q)). If F is a forecast and G a climatology, F(q) is the so-called crossing-point forecast (Ben Bouallègue, 2021). By analogy, q is called the crossing-point quantile, which is the focus of this study.

The concept of crossing-point forecast and quantile is illustrated in Figure 2(b). The single intersection point between the ensemble distribution (F, represented by a solid gray line) and the climatology distribution (G, represented by a dashed gray line) is indicated by a circle. CPF is the projection of this point on the y-axis and so takes a value between 0 and 1 (not shown). In other words, CPF is a quantile level that varies for each forecast. Consequently, the crossing-point quantile q is a self-adaptive-quantile forecast, a quantile forecast at a quantile level corresponding to the CPF of the day (indicated by a pink vertical line).

In the estimation of the CPF, forecast and climate distributions refer to the same space, the “model space.” In that case, the crossing-point quantile is denoted by $q_{c_{pf}}^M$ as it is based on the model climate. When issuing an adaptive-quantile forecast, one can consider using an observation climatology instead of a model climatology. To be consistent and to ensure forecast reliability, the ensemble can be calibrated in the “observation space.” The calibrated forecast and observation climatology can differ from the raw forecast and the model climatology, respectively, as illustrated in Figure 2(c). When based on a calibrated forecast and the observation climatology, the crossing-point quantile is denoted by $q_{c_{pf}}^O$.

More details about how model and observation climate distributions are built in our experiments can

be found in Appendix A while a short description of the methodology followed to calibrate the ensemble is provided in Appendix B.

## 2.3 THE OPTIMAL FORECAST IN TERMS OF PSS

The Peirce skill score (PSS) is a verification metric for dichotomous (yes/no) forecasts assessed against dichotomous observations (an event happens or not). We can show that the crossing-point quantile forecast is the optimal forecast in terms of PSS for any event.

Let us first define PSS. Consider a dichotomous forecast for a yes/no event (typically the observation being greater than a threshold). For verification, a contingency table is populated by counting the number of hits (A), false alarms (B), misses (C), and correct negatives (D). The hit rate (HR) is defined as the ratio $A/(A+C)$ while the false alarm rate (FAR) is defined as the ratio $B/(B+D)$. The PSS definition is as follows:

$$PSS = HR - FAR. \qquad (3)$$

A direct link exists between PSS and the relative operating characteristic (ROC) curve. For a dichotomous forecast, the ROC is defined by the point [FAR, HR]. In that case, Manzato (2006) showed that PSS and the area under the ROC curve (AUC) are related as follows:

$$PSS = 2AUC - 1. \qquad (4)$$

The right-hand side of Eq. (4) is also known as the ROC skill score.

Now, consider a forecast for a binary event that takes the form of a probability. In that case, the ROC curve is a set of points [FAR, HR] estimated for a range of probability thresholds. A theoretical ROC curve for a probability forecast is illustrated in Figure 3(a). In this situation, the point closer to the left top corner of the plot corresponds to the probability threshold that maximizes PSS (i.e., the difference between HR and FAR). If the forecast is reliable, it can be shown that the tangent to the ROC curve at

<figure><img src="image_6.png" /><figcaption>Figure 3 (a) Relative operating characteristic (ROC) curve for a probability forecast where each point corresponds to results for a different probability threshold. The blue square corresponds to the probability threshold that maximizes the Peirce skill score (PSS). (b) Area under the ROC curve for the optimal probability threshold is shown in gray while the corresponding PSS is indicated by a blue dashed vertical line.</figcaption></figure>
that point has a slope equal to 1 and that this point corresponds to the results obtained with a probability threshold equal to the event base rate (Ben Bouallègue et al., 2015; Richardson, 2011). This latter characteristics (the optimal probability threshold is equal to the event base rate) is key in the following demonstration.

Now, coming back to the crossing-point forecast, let us define more formally an event of interest, say $x > \theta$ with $\theta$ the event threshold (e.g., rain observed greater than 15 mm/24 h). The base rate of this event, denoted by $p_c$, is by definition equal to $1 - G(\theta)$, where G is the climate cumulated probability distribution. We denote $p_f$ as the probability forecast for this event, which is by definition $1 - F(\theta)$, where F is the forecast cumulated probability distribution. Using Eqs (1) and (2), we can derive the following:

$$p_f < p_c \Rightarrow 1 - F(\theta) < 1 - G(\theta) \Rightarrow F(\theta) > G(\theta) \Rightarrow \theta \ge q \quad (5)$$

and

$$p_f \ge p_c \Rightarrow 1 - F(\theta) \ge 1 - G(\theta) \Rightarrow F(\theta) \le G(\theta) \Rightarrow \theta < q \quad (6)$$

In plain words, Eqs (5) and (6) mean that taking action when the probability forecast $p_f$ is greater than the event base rate $p_c$ is equivalent to taking action when the crossing-point quantile $q$ exceeds the event threshold $\theta$. The former corresponds to the optimal choice to maximize PSS. So, conditioned on reliability, the crossing-point quantile is the optimal forecast in terms of PSS (and equivalently in terms of ROC area) for any dichotomous event defined by a threshold.

## 3 A QUALITATIVE ASSESSMENT

We now assess qualitatively the different types of point-forecasts using case studies. For this purpose, we show two types of plots: (1) time series of precipitation maxima within an area over consecutive summer days and (2) maps of precipitation fields for an interesting date in the series.

The observation used to compute statistics, build a climatology, and illustrate this article is a radar dataset called COMEPHORE (Combination for Best Estimation of Hourly Precipitation). This hourly precipitation reanalysis product is based on radar and rain gauge data, covering metropolitan France (Tabary et al., 2012).

The data, originally on a 1 km grid, are aggregated at the spatial resolution of the ensemble forecast, that is, 18 km. The upscaling of the observation consists in averaging the precipitation at the new spatial scale. This step is applied consistently in the verification and in the observation climatology. This choice aims at a fair comparison of the different point-forecasts evaluated here, all interpreted as averaged precipitation over a

grid box. For operational purposes, one could instead consider using a COMEPHORE climatology at the original scale to benefit from a downscaling while estimating the crossing-point quantile.

### 3.1 TIME SERIES

We define an area of 10 × 10 model grid points centered over Toulouse, France. For each forecast as well as for the radar, the maximum precipitation within this area for each day is shown in Figure 4. The time series cover June, July, and August (JJA) 2022.

The ensemble mean (em) systematically underestimates the precipitation amount, but larger em values (in relative terms) are generally associated with larger observations. The crossing-point quantiles, both $q_{c_{pf}}^M$ and $q_{c_{pf}}^O$, show a good fit with the observations, with the former ($q_{c_{pf}}^O$) displaying larger values that match better with the observations in this example. With the control member (ctrl), several large precipitation events are missed while the ensemble maximum (mx) is more prone to false alarms with sometimes large forecast values when no rain is observed. Finally, the conditional quantile ($q_{c_{70}}^O$) shows a systematic underestimation of large precipitation events.

The most extreme precipitation event observed in the series was on August 14 (day 75). This event seems particularly well captured by the crossing-point quantiles with an intensity closer to the observation for $q_{c_{pf}}^O$. We now assess visually the forecasts as fields valid on that particular date.

### 3.2 MAPS

The case study discussed here refers to August 14, 2024, identified as the most extreme event during the summer of 2024 in the Toulouse area. The different point-forecasts are shown in Figure 5(a) to 5(f), all have a forecast lead time of 5 days. In Figure 6(a) to 6(e), we focus on the crossing-point quantile forecasts $q_{c_{pf}}^O$ and compare forecasts at decreasing lead times but with the same validity time.

The radar observation for the verification date is shown at its native resolution in Figure 6(f). The radar captured the strongest precipitation signal over the South West of France with values exceeding 80 mm/24 h. High precipitation values are also registered along a South-West/North-East axis with larger scale precipitation in the North-East of France.

In this example, the crossing-point quantiles shown in Figure 5(a) and 5(b) exhibit heavy precipitation forecasts over the whole of France except Brittany and the South-East, with larger values in $q_{c_{pf}}^O$. In Figure 5(c), the conditional quantile shows a similar picture but with an intensity typically twice smaller than $q_{c_{pf}}^M$. As shown in Figure 5(d), the maximum of all members forecast more than 5 mm/24 h of rain over the whole country with large value (greater than 40 mm/24 h) all along
<figure><img src="image_7.png" /><figcaption>**Figure 4** Time series of point-forecasts with corresponding observation over summer 2024. The maximum precipitation over a 10 × 10 grid-point area centered over Toulouse, France, is shown. The day indicated on the x-axis corresponds to the forecast starting date. All forecasts have a lead time of 5 days: (a) ensemble mean em, (b) adaptive-quantile with model climate $q_{cpf}^M$, (c) adaptive-quantile with radar climate $q_{cpf}^O$, (d) control member ctrl, (e) maximum of all ensemble members mx, and (f) conditional 70%-quantile $q_{70}^C$. On each plot, the radar observation is represented by two dashed curves corresponding to the mean and maximum value within the area.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>**Figure 5** Point-forecasts at day 5 valid on August 14, 2024: (a) crossing-point quantile based on the radar climatology, (b) crossing-point quantile based on the model climatology, (c) conditional 70%-quantile forecast, (d) maximum of all ensemble members, (e) ensemble mean, and (f) control forecast.</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>**Figure 6** Crossing-point quantile forecasts for consecutive runs (a) to (e), all valid on August 14, 2024, and (f) the corresponding radar observation.</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>**Figure 7** Observation and point-forecasts distributions at (a) day 1 and (b) day 5.</figcaption></figure>

a South-West/North-East axis. As shown in Figure 5(e) and 5(f), the ensemble mean and control forecast, respectively, performed poorly at this lead time with generally lower precipitation rates than in the radar and missed events in the North-East.

Figure 6(a) to 6(e) shows the evolution of a crossing-point quantile forecast over consecutive runs. Typically, the adaptive-quantile forecast would indicate larger areas with potentially extreme events at longer lead times (when the forecast uncertainty is large). As we approach the time of the event, the forecast is sharper with smaller areas indicating the potential occurrence of an extreme event. The forecast also appears less noisy at shorter lead times. More ensemble members would allow better capturing the tail of the forecast distribution and thus help reduce the forecast noise in the medium range.

## 3.3 FORECAST CONSISTENCY

Is one type of forecast generally overestimating or underestimating the observed precipitation? The examples shown in Figures 4 and 5 seem to indicate that this is the case but to answer this question systematically, we compare the distributions of the forecast and the observation. Here, we define equally spaced bins and count the number of cases falling in each bin over the verification period. The results are shown in Figure 7.

In Figure 7(a), the forecast distributions at day 1 generally resemble the observation distribution with two striking exceptions: the ensemble mean (em) tends to systematically underestimate the highest precipitation while the maximum of all ensemble members (mx) tends to systematically overestimate it. It is worth noting that the control forecast (ctrl) shows good consistency with the observations. Here, the control forecast is the only
*point-forecast* that is also a coherent scenario (in space and time).

As a consequence, at day 5, as shown in Figure 7(b), the control forecast is the only forecast that still has some resemblance with the observation in terms of distribution. The other *point-forecast* distributions are somehow modulated by the uncertainty in the forecast at that time range. Ensemble mean (em) and conditional quantile ($q_{70}^C$) show a clear underestimation of larger precipitation values while the ensemble maximum tends to underestimate low precipitation amounts. Focusing on the tail of the observation distribution, we note that only the self-adaptive quantiles and the maximum of all members are able to generate the highest amount of precipitation observed in the radar (above 100 mm/24 h).

# 4 A QUANTITATIVE ASSESSMENT

A quantitative assessment complements the qualitative comparison based on case studies and the consistency check presented in Section 3. In this section, we discuss averaged scores computed for three consecutive summers, that is June, July, and August in 2022, 2023, and 2024.

## 4.1 OPTIMAL POINT-FORECASTS

The first result was already presented in Figure 1: the ensemble mean is the best forecast in terms of RMSE (Figure 1a) while the ensemble maximum is the best forecast in terms of QSmax (Figure 1b). The crossing-point quantile forecasts $q_{cpf}^M$ and $q_{cpf}^O$ are “intermediate” forecasts for both metrics with a larger variability in $q_{cpf}^O$ leading to worse RMSE performance.

In theory, the crossing-point forecast should be the optimal forecast in terms of PSS for any event as discussed in Section 2.3. Whether this property holds in practice is explored in Figure 8. The PSS (equivalently the

AUC) are shown for a range of event thresholds from 0.1 mm/24 h up to 40 mm/24 h and for a range of forecast lead times from day 1 to day 7. Figure 8(a) focuses on one lead time (day 5) while Figure 8(b) focuses on one event threshold (15 mm/24 h).

Figure 8 supports our theoretical findings: the crossing-point forecast is the best or among the best forecasts compared here in terms of PSS (within score uncertainty margins). This result holds for all lead times. Importantly, we note that calibration plays a key role to reach this level of skill: calibration is a necessary condition for optimality.

Focusing on day 5, Figure 9 shows ROC curves for two events: precipitation exceeding 1 mm/24 h and 15 mm/24 h in Figure 9(a) and 9(b), respectively. Typically, the ensemble maximum has a higher false alarm rate than optimal for smaller thresholds while the ensemble mean and control forecast tends to have a smaller hit rate than optimal for higher thresholds. Also, the impact of calibration is visible in Figure 9(b) with a ROC point for $q_{cpf}^O$ closer to the top-left corner of the plot than $q_{cpf}^M$: calibration is needed to ensure optimality for higher event thresholds.

## 4.2 ECONOMIC VALUE

The economic value of a forecast is estimated here with the help of a simple cost-loss model. We consider the case where a forecast user can mitigate the loss associated with the occurrence of a weather event by taking a preventive action. The user decides to take action or not based on the forecast. The key parameters of this model are (1) the cost of taking action and (2) the loss encountered if no protective action is taken but the event occurs. The cost-loss ratio characterizes the user in the sense that it reflects a specific user-defined application. Building on this model, the so-called potential economic value is derived from the elements of the contingency table (Richardson, 2000).

<figure><img src="image_11.png" /><figcaption>**Figure 8** Peirce skill score (PSS) or equivalently area under the relative operating characteristic (ROC) curve (AUC) skill score (a) as a function of the event threshold for a lead time of 5 days and (b) as a function of the lead time for an event threshold of 15 mm/24 h.</figcaption></figure>
<figure><img src="image_12.png" /><figcaption>**Figure 9** Relative operating characteristic (ROC) curves at day 5 for an event threshold of (a) 1 mm/24 h and (b) 15 mm/24 h.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>**Figure 10** Same as Figure 9 but for the economic value.</figcaption></figure>

Figure 10 compares the economic value of different types of point-forecasts as a function of the user's cost-loss ratio. Figure 10(a) and 10(b) focus on two different events defined as precipitation exceeding 1 mm/24 h and 15 mm/24 h, respectively. Both plots are results for forecasts at a lead time of 5 days. Note that a logarithmic scale is used for the x-axis in Figure 10 that put more emphasis on low cost-loss ratios. The ensemble maximum has a higher economic value for very low cost-ratios, and adaptive-quantile forecast has a higher economic value than other forecasts for cost-loss ratios in the range of the event base-rate (indicated by a vertical line). For applications with larger cost-loss ratios, the conditional quantile and ensemble mean are better choices for decision-making.

# 5 SUMMARY AND OUTLOOK

The crossing-point is the intersection point between two cumulative probability distributions: a forecast and a climatology. The corresponding quantile varies with the forecast uncertainty of the day. This self-adaptive-quantile forecast is called the crossing-point quantile forecast. In our study, we discussed the properties and performance of this new type of point-forecast. Our summary points and outlook are as follows:

* **Elicitation.** The link between crossing-point forecast and PSS is similar to the link between mean forecast and RMSE, or median forecast and MAE. Indeed, we demonstrate that the crossing-point quantile is the
optimal forecast in terms of PSS (and equivalently in terms of ROC area) for any event. Furthermore, the crossing-point quantile links the PSS with the Diagonal score, which can be interpreted as a weighted average of AUC, as shown in Appendix C.

* **Extreme events.** Optimality for any event includes optimality for rare and extreme events. We showcase the crossing-point performance with case studies and statistical analysis that support our theoretical findings. The predictive power of the crossing-point forecast for extreme events is also explored in Ben Bouallègue (2024).

* **Condition of optimality.** The necessary condition for optimality is calibration. The ensemble or probabilistic forecast needs to be well-calibrated to ensure a theoretical guarantee of optimality. When necessary, statistical post-processing can be applied to reach that goal (Vannitsem et al., 2021).

* **Model climate.** When post-processing is used to calibrate the forecast, there is no need to build an M-climate based on reforecasts (which can be expensive). After calibration in the observation space, the climate distribution used to build the CPF would be directly estimated from observations for consistency. In the case of a well-calibrated system, model and observation have the same climatology.

* **Observation climatology.** The estimation of climate percentiles can be a moving target due to natural variability and anthropogenic climate change. When one build a climatology with limited observation records, one could consider using extreme value theory to better capture the tail of the distribution. For example, non-negative precipitation amounts can be fitted with an Extended Pareto Distribution as suggested by Naveau et al. (2016).

* **Interpretation.** Like any other quantile forecast, a self-adaptive-quantile forecast based on the CPF is not a physically consistent spatial or temporal scenario. A crossing-point quantile forecast should be interpreted at each location (grid-point) as a local “probabilistic worst-case scenario” (Ben Bouallègue, 2021). In simple terms, the users are dealing with a state-dependent quantile, where the quantile level becomes more extreme as the predicted probability of an extreme event increases, a situation already observed in practical cases (Schmidt et al., 2021).

* **Usage.** Self-adaptive-quantile forecasts are particularly well-suited for users whose cost-loss ratio is unfocused, that is, with a cost-loss ratio that decreases as the rarity of the event increases. As revealed by our economic value analysis, the crossing-point quantile forecast is not suitable for all users. In particular, users sensitive to false alarms might consider using the ensemble mean instead,

which displays large values only when the forecast uncertainty is small.

* **Loss function.** In principle, weighted proper scores could be used as loss functions to train weather forecasting models based on machine learning. For example, one could consider the Diagonal score which is consistent with the crossing-point forecast as a performance metric. The link between Diagonal score, PSS, and the coefficient of predictive ability (CPA, Gneiting and Walz, 2022) is developed in Appendix C.

* **Outlook.** The quality of the CPF predominantly depends on the quality of the underlying ensemble prediction system. So, improvement for this new type of prediction would lie in (1) better raw (or post-processed) ensemble forecasts and (2) more ensemble members for a finer estimation of the intersection point between distributions. Machine learning could help achieve these objectives in a near future.

# APPENDIX

## A MODEL AND RADAR CLIMATOLOGY

The model climatology, also known as M-climate, is built on 20 years of reforecasts. A period of 5 weeks centered around the current day is used to account for seasonal variations as described in Lalaurette (2003). Similarly, an observation climatology is estimated for each week of the year using a 5-week rolling period over 20 years of records from 2001 to 2020.

## B ENSEMBLE CALIBRATION

Ensemble calibration is based on a member-by-member approach (MBM, Van Schaeybroeck and Vannitsem, 2015) and tested successfully on ensemble precipitation forecasts (e.g., Ben Bouallègue et al., 2024). A correction is applied to each ensemble member individually with a component common to all members and a component that adjusts the deviation of a member with respect to the ensemble mean. If we denote $\hat{x}_i$ the corrected forecast for the mth member of the ensemble, MBM formally consists in applying:

$$ \hat{x}_i = \alpha + \beta \bar{x} + \gamma(x_i - \bar{x}), \quad (7) $$

where $x_i$ is the ensemble member i and $\bar{x}$ is the ensemble mean. A clipping to zero is applied to the corrected forecast. The parameter $\alpha$ is the bias parameter that nudges the ensemble mean, $\beta$ is the linear coefficient that scales the ensemble mean, and $\gamma$ is the scaling parameter that adjusts the spread of the ensemble. The coefficients $\alpha$, $\beta$, and $\gamma$ are estimated over the summer of 2021 for all grid points together but for each forecast lead time independently.
## C DIAGONAL SCORE, PEIRCE SKILL SCORE, AND UNIVERSAL ROC

Following Ben Bouallègue (2021), the Diagonal score $D_G$ is defined using elementary scores $d_G$ as follows:

$$D_G(F, y) := \int_0^1 d_G(F^{-1}(1-\alpha), y) d\alpha \quad (8)$$

with G the climatology probability distribution, F the forecast probability distribution, and

$$d_G(F^{-1}(1-\alpha), y) = \begin{cases} 1-\alpha & \text{if } y > G^{-1}(1-\alpha) \ge F^{-1}(1-\alpha) \\ \alpha & \text{if } F^{-1}(1-\alpha) > G^{-1}(1-\alpha) \ge y \\ 0 & \text{otherwise} \end{cases} \quad (9)$$

When the observation ranges the unconditional distribution G, the expectation of the elementary Diagonal score is a function of PSS:

$$\mathbb{E}[d_G(F^{-1}(1-\alpha), Y)] = \alpha(1-\alpha)[1-(H_\alpha - F_\alpha)], \quad (10)$$

with $H_\alpha = \mathbb{P}(F^{-1}(1-\alpha) > G^{-1}(1-\alpha)|Y > G^{-1}(1-\alpha))$ and $F_\alpha = \mathbb{P}(F^{-1}(1-\alpha) > G^{-1}(1-\alpha)|Y \le G^{-1}(1-\alpha))$, respectively the hit rate and the false alarm rate associated to the event $G^{-1}(1-\alpha)$. For the climatological forecast, $\mathbb{E}[d_G(G^{-1}(1-\alpha), Y)] = \alpha(1-\alpha)$.

In this framework, PSS can be written as the elementary diagonal skill score:

$$H_{\alpha} - F_{\alpha} = 1 - \frac{\mathbb{E}[d_G(F^{-1}(1-\alpha), Y)]}{\mathbb{E}[d_G(G^{-1}(1-\alpha), Y)]} \quad (11)$$

Similarly, the expected Diagonal score can be understood as a sum of quadratic weighted PSS:

$$\mathbb{E}[D_G(F, Y)] = \int_0^1 \alpha(1-\alpha)[1-(H_\alpha - F_\alpha)] d\alpha. \quad (12)$$

Thus the expected Diagonal score of a climate forecast G is 1/6. This score is obtained for any non-informative forecast, that is, a forecast for which $H_\alpha = F_\alpha$. In other words, the Diagonal score is equitable.

Moreover, Gneiting and Walz (2022) defined the CPA as a general measure of potential predictive ability. CPA takes value between 1/2 and 1 and is equivalent to the area under the universal ROC, obtained by convex combinations of ROC curves for any event of interest. In the case of point-forecasts, the CPA can be informally defined as follows, up to a multiplicative factor:

$$2CPA - 1 = \int_0^1 6\alpha(1-\alpha)(H_\alpha - F_\alpha) d\alpha \\ = 1 - 6\mathbb{E}[D_G(F, Y)]. \quad (13)$$

As a result, the general measure of predictive ability of Gneiting and Walz (2022) is equivalent to the Diagonal skill score:

$$2CPA - 1 = 1 - \frac{\mathbb{E}[D_G(F, Y)]}{\mathbb{E}[D_G(G, Y)]} \quad (14)$$

The crossing-point quantile is also elicitable with respect to the same scoring function (the Diagonal score) as being a differentiable transformation of the crossing-point forecast. This result is derived from the work of Osband (1985), later recalled by Gneiting (2011) and Fissler and Ziegel (2016) among others.

## NOTE

1. If one would like to optimize the continuous ranked probability score, the ensemble members must be interpreted as quantile forecasts at probability levels $\frac{m-0.5}{M}$ with $m \in \{1, \dots, M\}$ according to Bröcker (2012).

## ACKNOWLEDGEMENTS

The authors are grateful to an anonymous reviewer for their insightful and constructive comments. Also, MT would like to thank the University of Reading for hospitality and the Isaac Newton Institute for Mathematical Sciences for support during the satellite program “Geophysical fluid dynamics; from mathematical theory to operational prediction” when work on this paper was undertaken.

## FUNDING INFORMATION

MT’s contribution to this work was supported by EPSRC grant number EP/R014604/1.

## COMPETING INTERESTS

The authors have no competing interests to declare.

## AUTHOR AFFILIATIONS

Zied Ben Bouallègue [ORCID](https://orcid.org/0000-0003-2914-4203)

The European Centre for Medium-Range Weather Forecasts, Reading, UK

Maxime Taillardat [ORCID](https://orcid.org/0000-0003-2039-4476)

CNRM, Université de Toulouse, Météo-France, CNRS, Toulouse, France

## REFERENCES

* Bakhshaii, A. and Stull, R. (2009) Deterministic ensemble forecasts using gene-expression programming. Weather and Forecasting, 24(5): 1431–1451. DOI: https://doi.org/10.1175/2009WAF2222192.1

* Ben Bouallègue, Z. (2021) On the verification of the crossing-point forecast. Tellus A: Dynamic Meteorology and Oceanography, 73(1): 1–10. DOI: https://doi.org/10.1080/16000870.2021.1913007
**Ben Bouallègue, Z.** (2024) Seamless prediction of high-impact weather events: A comparison of actionable forecasts. *Tellus A: Dynamic Meteorology and Oceanography*, 76(1): 14–28. DOI: https://doi.org/10.16993/tellusa.3262

**Ben Bouallègue, Z., Haiden, T.** and **Richardson, D.S.** (2018) The Diagonal score: Definition, properties, and interpretations. *Quarterly Journal of the Royal Meteorological Society*, 144(714): 1463–1473. DOI: https://doi.org/10.1002/qj.3293

**Ben Bouallègue, Z., Pinson, P.** and **Friederichs, P.** (2015) Quantile forecast discrimination ability and value. *Quarterly Journal of the Royal Meteorological Society* 141: 3415–3424. DOI: https://doi.org/10.1002/qj.2624

**Ben Bouallègue, Z., Weyn, J.A., Clare, M.C.A., Dramsch, J., Dueben, P.** and **Chantry, M.** (2024) Improving medium-range ensemble weather forecasts with hierarchical ensemble transformers. *Artificial Intelligence for the Earth Systems*, 3(1): e230 027. DOI: https://doi.org/10.1175/AIES-D-23-0027.1

**Bjerknes, V.** (1904) Das Problem der Wettervorhersage, betrachtet vom Standpunkte der Mechanik und der Physik. *Meteorologische Zeitschrift* 21: 1–7.

**Bouttier, F.** and **Marchal, H.** (2024) Probabilistic short-range forecasts of high precipitation events: Optimal decision thresholds and predictability limits. *EGUSphere*, 2024: 1–30. DOI: https://doi.org/10.5194/nhess-24-2793-2024

**Bright, D.R.** and **Nutter, P.A.** (2004) Identifying the “best” ensemble member. *Bulletin of the American Meteorological Society*, 85(1): 13–13.

**Bröcker, J.** (2012) Evaluating raw ensembles with the continuous ranked probability score. *Quarterly Journal of the Royal Meteorological Society*, 138(667): 1611–1617. DOI: https://doi.org/10.1002/qj.1891

**Chen, L.** (2024) A review of the applications of ensemble forecasting in fields other than meteorology. *Weather*, 79(9): 285–290. DOI: https://doi.org/10.1002/wea.4584

**Cooke, E.** (1906) Forecasts and verifications in Western Australia. *Monthly Weather Review*, 34(1): 23–24. DOI: https://doi.org/10.1175/1520-0493(1906)34<23:FAVIWA>2.0.CO;2

**Fissler, T.** and **Ziegel, J.F.** (2016) Higher order elicitability and osband’s principle. *Annals of Statistics*, 44(4): 1680–1707. DOI: https://doi.org/10.1214/16-AOS1439

**Fundel, V.J., Fleischhut, N., Herzog, S.M., Göber, M.** and **Hagedorn, R.** (2019) Promoting the use of probabilistic weather forecasts through a dialogue between scientists, developers and end-users. *Quarterly Journal of the Royal Meteorological Society*, 145: 210–231. DOI: https://doi.org/10.1002/qj.3482

**Gneiting, T.** (2011) Making and evaluating point forecasts. *Journal of the American Statistical Association*, 106(494): 746–762. DOI: https://doi.org/10.1198/jasa.2011.r10138

**Gneiting, T.** and **Walz, E.M.** (2022) Receiver operating characteristic (ROC) movies, universal ROC (UROC) curves, and coefficient of predictive ability (CPA). *Machine Learning*,

111(8): 2769–2797. DOI: https://doi.org/10.1007/s10994-021-06114-3

**Joslyn, S.** and **Savelli, S.** (2010) Communicating forecast uncertainty: Public perception of weather forecast uncertainty. *Meteorological Applications*, 17(2): 180–195. DOI: https://doi.org/10.1002/met.190

**Lalaurette, F.** (2003) Early detection of abnormal weather conditions using a probabilistic extreme forecast index. *Quarterly Journal of the Royal Meteorological Society*, 129(594): 3037–3057. DOI: https://doi.org/10.1256/qj.02.152

**Leith, C.E.** (1974) Theoretical skill of Monte Carlo forecasts. *Monthly Weather Review*, 102(6): 409–418. DOI: https://doi.org/10.1175/1520-0493(1974)102<0409:TSOMCF>2.0.CO;2

**Manzato, A.** (2006) A note on the maximum Peirce skill score. *Weather and Forecasting*, 22: 1148–1154. DOI: https://doi.org/10.1175/WAF1041.1

**Morss, R.E., Lazo, J.K., Brown, B.G., Brooks, H.E., Ganderton, P.T.** and **Mills, B.N.** (2008) Societal and economic research and applications for weather forecasts: Priorities for the North American Thorpex program. *Bulletin of the American Meteorological Society*, 89(3): 335–346. DOI: https://doi.org/10.1175/BAMS-89-3-335

**Naveau, P., Huser, R., Ribereau, P.** and **Hannart, A.** (2016) Modeling jointly low, moderate, and heavy rainfall intensities without a threshold selection. *Water Resources Research*, 52(4): 2753–2769. DOI: https://doi.org/10.1002/2015WR018552

**Ono, K.** (2023) Clustering technique suitable for Eulerian framework to generate multiple scenarios from ensemble forecasts. *Weather and Forecasting*, 38(6): 833–847. DOI: https://doi.org/10.1175/WAF-D-22-0137.1

**Osband, K.H.** (1985) Providing incentives for better cost forecasting. PhD thesis, University of California.

**Pagano, T.C., Pappenberger, F., Wood, A.W., Ramos, M.H., Persson, A.** and **Anderson, B.** (2016) Automation and human expertise in operational river forecasting. *Wiley Interdisciplinary Reviews: Water*, 3(5): 692–705. DOI: https://doi.org/10.1002/wat2.1163

**Peirce, C.** (1884) The numerical measure of the success of predictions. *Science*, 4: 453–454. DOI: https://doi.org/10.1126/science.ns-4.93.453.b

**Richardson, D.S.** (2000) Skill and relative economic value of the ECMWF ensemble prediction system. *Quarterly Journal of the Royal Meteorological Society*, 126: 649–667. DOI: https://doi.org/10.1002/qj.49712656313

**Richardson, D.S.** (2011) Economic value and skill. In: Jolliffe, I.T. and Stephenson, D.B. (eds.). *Forecast Verification: A Practitioner’s Guide in Atmospheric Science*. John Wiley & Sons, pp. 167–184. DOI: https://doi.org/10.1002/9781119960003.ch9

**Roebber, P.J.** (2010) Seeking consensus: A new approach. *Monthly Weather Review*, 138(12): 4402–4415. DOI: https://doi.org/10.1175/2010MWR3508.1
Roulston, M.S. and Smith, L.A. (2003) Combining dynamical and statistical ensembles. *Tellus A: Dynamic Meteorology and Oceanography*, 55(1): 16–30. DOI: https://doi.org/10.3402/tellusa.v55i1.12082

Roulston, M.S. and Smith, L.A. (2004) The boy who cried wolf revisited: The impact of false alarm intolerance on cost-loss scenarios. *Weather and Forecasting*, 19(2): 391–397. DOI: https://doi.org/10.1175/1520-0434(2004)019<0391:TBWCWR>2.0:CO2

Schmidt, P., Katzfuss, M. and Gneiting, T. (2021) Interpretation of point forecasts with unknown directive. *Journal of Applied Econometrics*, 36(6): 728–743. DOI: https://doi.org/10.1002/jae.2833

Schwartz, C.S., Romine, G.S., Smith, K.R. and Weisman, M.L. (2014) Characterizing and optimizing precipitation forecasts from a convection-permitting ensemble initialized by a mesoscale ensemble Kalman filter. *Weather and Forecasting*, 29(6): 1295–1318. DOI: https://doi.org/10.1175/WAF-D-13-00145.1

Tabary, P., Dupuy, P., L’Henaff, G., Gueguen, C., Moulin, L., L’Lantin, O., Merlier, C. and Soubeyroux, J.-M. (2012) A 10-year (1997–2006) reanalysis of Quantitative Precipitation Estimation over France: Methodology and first results. *Weather Radar and Hydrology*, 1(351): 255–260.

Van Schaeybroeck, B. and Vannitsem, S. (2015) Ensemble post-processing using member-by-member approaches: Theoretical aspects. *Quarterly Journal of the Royal Meteorological Society*, 141(688): 807–818. DOI: https://doi.org/10.1002/qj.2397

Vannitsem, S., Bremnes, J.B., Demaeyer, J., Evans, G.R., Flowerdew, J., Hemri, S., Lerch, S., Roberts, N., Theis, S., Atencia, A. et al. (2021) Statistical postprocessing for weather forecasts: Review, challenges and avenues in a big data world. *Bulletin of the American Meteorological Society*, 102(3): E681–E699. DOI: https://doi.org/10.1175/BAMS-D-19-0308.1

Ziegel, J.F. (2016) Coherence and elicitability. *Mathematical Finance*, 26(4): 901–918. DOI: https://doi.org/10.1111/mafi.12080

**TO CITE THIS ARTICLE:**

Ben Bouallègue, Z. and Taillardat, M. (2025) Self-Adaptive Quantiles for Precipitation Forecasting. *Tellus A: Dynamic Meteorology and Oceanography* 77(1): 160–172. DOI: https://doi.org/10.16993/tellusa.4110

Submitted: 28 March 2025

Accepted: 27 May 2025

Published: 12 June 2025

**COPYRIGHT:**

© 2025 The Author(s). This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC-BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. See http://creativecommons.org/licenses/by/4.0/.

*Tellus A: Dynamic Meteorology and Oceanography* is a peer-reviewed open access journal published by Stockholm University Press.

<figure><img src="image_14.png" /></figure>

<figure><img src="image_15.png" /></figure>