# Texture-based classification of high-resolution precipitation forecasts with machine-learning methods

Yamina Hamidi, Laure Raynaud, Lucie Rottner, Philippe Arbogast

► To cite this version:

Yamina Hamidi, Laure Raynaud, Lucie Rottner, Philippe Arbogast. Texture-based classification of high-resolution precipitation forecasts with machine-learning methods. Quarterly Journal of the Royal Meteorological Society, 2020, 146 (732), pp.3014-3028. 10.1002/qj.3823. hal-04918253

HAL Id: hal-04918253

https://cnrs.hal.science/hal-04918253v1

Submitted on 23 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
<figure><img src="image_2.png" /></figure>

DOI: 10.1002/qj.3823

RESEARCH ARTICLE

Quarterly Journal of the
Royal Meteorological Society

<figure><img src="image_3.png" /></figure>

RMets

# Texture-based classification of high-resolution precipitation forecasts with machine-learning methods

Yamina HamidiID |

Laure RaynaudID |

Lucie Rottner |

Philippe Arbogast

CNRM UMR 3589, Météo-France/CNRS,
Toulouse, France

## Correspondence

Y. Hamidi, Météo-France CNRM/GMAP,
42 Avenue G. Coriolis, 31057 Toulouse
Cedex, France.

Email: yamina.hamidi@meteo.fr

## Abstract

Object-based methods are commonly used for verification and postprocessing of high-resolution precipitation forecasts. They usually detect objects based on intensity criteria only, without considering the spatial organization of rainfall, known as texture. This article evaluates the performance of several machine-learning methods to detect “continuous” and “intermittent” rainfall patterns in the forecasts of the French convective-scale Arome model. A sliding-window segmentation algorithm, which applies a classification model at each grid point, is implemented. Several classifiers and input textural features are compared. Overall, intermittent precipitation is the most difficult to detect. The random forest classifier is shown to provide the best classification results independently of the predictor used, with a surprising ability to extract a relevant signal from a synthetic descriptor such as the rainfall cumulative distribution function, as well as from the large amount of unprocessed information provided by neighbouring grid points. On the other hand, the logistic regression classifier needs a texture-oriented predictor, such as the statistics derived from the grey-level co-occurrence matrix, to perform well. Global insight into model behaviour is then obtained by examining the importance of input features. Finally, we show that random forests trained on Arome deterministic forecasts can be applied successfully to discriminate between precipitation textures in different Arome configuration outputs and gridded observations.

## KEYWORDS

computer vision, machine learning, rainfall texture, supervised classification

# 1 | INTRODUCTION

Convection-permitting numerical weather prediction (NWP) models are now widely used for operational forecasting, since they often provide realistic predictions of small-scale high-impact weather phenomena such as thunderstorms and heavy rainfall (Ebert, 2008). Nonetheless, due to the limited predictability of these events, their exact location and timing are still difficult to forecast, even at short ranges (Arbogast et al., 2016).

In this context, relevant information may be extracted at a larger and thus more predictable scale. For that purpose, several spatialization methods have been proposed (Gilleland et al., 2009). For instance, neighbourhood processing introduces spatial and/or temporal tolerances to reduce the constraint of an exact match between forecasts and observations. Another possible approach is to consider a forecast as a field of coherent structures, also known as objects. Regarding precipitation, different algorithms for detecting precipitating objects can be found in the
literature, with applications to forecast verification (Ebert and McBride, 2000; Davis et al., 2006; Ebert, 2008; Marzban and Sandgathe, 2008) and model postprocessing (Arbogast et al., 2016, Rottner et al., 2019; Raynaud et al., 2019).

While commonly used object detection methods are often based on a thresholding of rainfall fields (e.g., Continuous Rain Areas [CRA] in Ebert 2008; Method for Object-based Diagnostic Evaluation [MODE] in Davis et al., 2006), a “fuzzy” detection, which takes into account uncertainty regarding the shape and location of the object, was proposed initially by Arbogast et al. (2016) and revised recently by Raynaud et al. (2019) and Rottner et al. (2019). This method is based on the comparison of local rainfall distributions with some prescribed reference distributions, and is hereafter referred to as the similarity-based method. It has been shown to work well for the identification of three types of precipitating object, corresponding to total rainfall, moderate rainfall, and heavy rainfall. The existing detection algorithms can provide a variety of precipitating objects; however, they all rely on intensity criteria.

The spatial organisation of precipitation, also known as texture, is another essential aspect that should be characterised in order to obtain a complete description of precipitation patterns. The goal of this article is to introduce and evaluate the performance of several texture classification methods for the processing of precipitation forecasts from the French convective-scale Arome model. To the best of our knowledge, this is the first time this challenge has been addressed.

Classical computer vision techniques will be used to characterise continuous and intermittent rainfall. In some cases, intermittent precipitation can be spatially well-structured with repeated patterns, but identifying these details is beyond the scope of this article.

Texture analysis methods can be classified into four categories (Tuceryan and Jain, 1993; Materka et al., 1998). Geometrical methods consider texture as the repetition of a primitive (texture element). They aim at identifying the primitive and the underlying rule behind its spatial organisation (Chen, 1993). Filtering methods perform a frequential and/or spatial analysis of the image with tools such as the Fourier transform, Gabor filters (Turner, 1986), or wavelet transforms (Mallat, 1989). Model-based texture analysis methods consider the image as a realisation of a random process that can be described with a probabilistic model such as Markov random fields (Yousefi and Kehtaranavaz, 2011). Finally, statistical methods analyse the spatial distribution of grey levels by computing local features at each point in the image, such as histograms (Lowitz, 1983) and co-occurrence matrices (Haralick et al., 1973). In this work, only statistical methods will be investigated.

Given a texture extraction method, its detection can be obtained using an appropriate classification algorithm.

Hence, the precipitation texture recognition problem consists of determining both the most relevant textural features and the best-suited classification algorithm. The similarity-based method applied to texture histograms will be compared with two supervised machine learning algorithms: logistic regression and random forests. These classifiers have been applied successfully to similar classification problems (e.g., land-cover classification in Rodriguez-Galiano et al. (2012), cloud detection in Drönnner et al. (2018), and dryline identification in Clark et al. (2015)). Machine-learning techniques may help overcome the limitations of standard algorithms, which are usually based on a set of simple rules, by estimating a relevant model from the statistical properties of sample data. To that end, large training and testing datasets are required to fit and assess the performance of machine-learning algorithms. For the purpose of this work, a unique database of hand-labelled texture precipitation patterns has been constructed using hundreds of Arome forecasts.

The article is organized as follows. Section 2 details the experimental setup. Sections 3 and 4 present the texture descriptors and the classification algorithms, respectively. The performance of the different descriptors and classifiers is then discussed in section 5. In section 6, the ability of the trained classifiers to generalize on precipitation outputs from different versions of the Arome model and on observations is discussed. Finally, conclusions and future work are discussed in section 7.

## 2 | EXPERIMENTAL SETUP

### 2.1 | Arome and Arome-EPS models

Texture classification algorithms used in this work are run on one-hour accumulated rainfall forecasts from the French deterministic and probabilistic Arome models.

Arome is a spectral, nonhydrostatic convective-scale model, which has been running operationally at Météo-France since December 2008 over the French territory (Seity et al., 2011) to provide two-day-ahead forecasts. The current operational configuration uses a 1.3-km horizontal grid spacing and 90 vertical levels, and the analysis step is performed with an hourly 3D-Var scheme that assimilates a comprehensive set of observations (Brousseau et al., 2016). Arome precipitation forecasts are postprocessed on two horizontal regular latitude-longitude grids with horizontal grid spacings of 2.5 and 1 km, respectively.

The Arome Ensemble Prediction System (EPS) is a 16-member ensemble based on a 2.5-km version of the Arome-France model. It takes its lateral boundary conditions from the global French EPS (Descamps et al., 2015),
and uses state-of-the-art perturbation methods to repre-
sent initial uncertainties and model errors (Bouttier *et al.*,
2012; Raynaud and Bouttier, 2016).

## 2.2 | Training and validation datasets

Since classifiers are fitted classically on a training set and evaluated on an independent validation set, a database of 260 Arome deterministic one-hr precipitation forecasts has been designed by manually selecting cases over the years 2017–2019. These cases have been carefully chosen in order to sample the variety of continuous and intermittent rainfall patterns properly. This database has been separated into a training set and a validation set, consisting of 180 and 80 forecasts respectively (a classic two-thirds/one-third partitioning: James *et al.*, 2014). Usually this split is performed randomly; however, in order to preserve the diversity of training and validation datasets, here it has been performed manually. In both datasets, each sample consists of a 1-hr precipitation forecast interpolated on a 2.5-km grid and the corresponding ground-truth per-pixel handmade texture labels. These labels have been obtained after drawing continuous and intermittent contour objects manually: all pixels within a continuous (respectively intermittent) objects are labelled as continuous (respectively intermittent). Pixels outside objects are considered as nonprecipitating. Hand labelling is a time-consuming process and the resulting texture labels are highly subjective, since there is no exact definition of precipitation texture. The impact of this subjectivity will be discussed further in section 5.4.

The 2.5-km grid is used to develop texture detection methods adapted to both Arome and Arome EPS. It may be mentioned that, by nature, this dataset is highly imbalanced: no precipitation, intermittent precipitation, and continuous precipitation represent 81, 14, and 5% of the training sample, respectively. Different solutions have been proposed to address this class imbalance problem, such as undersampling data from the majority class and/or oversampling data from the minority class, but they are not explored in this article.

# 3 | TEXTURE DESCRIPTORS

As explained in the Introduction, the first step of the
classification problem is to extract the statistical textural
features that will be used as predictors by the classifica-
tion algorithms. The different textural descriptors used in
this study are the precipitation histogram, the statistics
derived from the grey-level co-occurrence matrix, and the
raw precipitation field.

<figure><img src="image_4.png" /><figcaption>**FIGURE 1** CDFs for continuous (black) and intermittent (grey) rainfall (restricted to the range 0–5 mm for clarity), computed from the training dataset of 1-hr accumulated precipitation Arome forecasts</figcaption></figure>

3.1 | Texture histogram

The precipitation histogram is a first-order statistic commonly used in texture analysis (Lowitz, 1983). It measures the frequency of precipitation intensity, but without taking into account the spatial interactions between the pixels. Figure 1 shows the cumulative distribution functions (CDFs) computed from the training dataset, for continuous and intermittent Arome rainfall forecasts. The two distributions present significant differences, especially between 0 and 5 mm, with, as expected, a larger proportion of low values for intermittent precipitation. Hence, despite its simplicity, the CDF could provide interesting information about texture. Since the main differences occur in the first bins, the CDFs is computed with fine binning in the first 10 mm and coarser binning for larger intensities. The precipitation intervals used are (0, 0.1, 0.5, 1, 2, ..., 10, 20, ..., 150) mm, corresponding to 26 bins.

3.2 | Grey-level co-occurrence matrix
(GLCM)

The co-occurrence matrix is one of the earliest used and
most popular methods to extract textural features, with,
for instance, successful applications to cloud classification
(Welch *et al.*, 1988). It is a second-order histogram that
analyses the grey-level (in the present case, the precipita-
tion intensity) distribution of pairs of neighbouring pixels.
The GLCM describes the spatial organisation of the pix-
els by measuring the frequency of occurrence of pixel pairs
given a distance *d* and a direction *θ*.
<figure><img src="image_5.png" /></figure>

Let us consider an $N_x \times N_y$ image $I$ with $N_g$ grey levels. The GLCM is then a $N_g \times N_g$ matrix $P$, the elements $P(i,j)$ of which are given by

$$P(i,j)_{d,\theta} = \mathrm{card}\{(x,y) : I(x,y) = i, I(x+d \cos(\theta), y+d \sin(\theta)) = j\}$$

where $i,j \in \{1,2, ..., N_g\}$. The GLCM can be computed in a single direction or in multiple directions that can be combined. In this work, the GLCM is averaged over eight directions $\theta \in \{0^\circ, 180^\circ, \pm 45^\circ, \pm 90^\circ, \pm 135^\circ\}$.

Since the number of precipitation intensity levels can be very large, the GLCM can be computationally expensive. For this reason, precipitation intensity values are quantized to a limited number of grey-value levels. If not defined properly, such quantization may lead to a loss of textural information. So, in this work, the quantization used to compute the GLCM is the one that has been defined for the CDFs (section 3.1).

To summarize the co-occurrence information, Haralick et al. (1973) proposed computing several statistics from the GLCM, including, for instance, contrast, homogeneity, energy, entropy, and correlation measures. Only the first 13 out of the 14 Haralick statistics have been used, due to the instability of computation of the 14th (Gandomi et al., 2015). Notations are those used by Haralick et al. (1973). The interested reader is referred to Bankman (2008) and Hall-Beyer (2017) for a detailed description of the statistics. Figure 2 shows the distributions of the 13 GLCM statistics used for the continuous and intermittent textures extracted from the Arome rainfall forecasts training dataset. According to the Kolmogorov-Smirnov distance, the statistics $f_1$, $f_6$, and $f_{13}$ present the largest differences between the continuous and intermittent samples. The statistics $f_1$ and $f_6$ measure homogeneity and $f_{13}$ measures correlation. In contrast, the discrimination ability of the statistic $f_2$, which measures contrast, is the lowest.

### 3.3 | Raw pixels

The last descriptor is given directly by the raw image pixels, without any preprocessing. Hence, there is no loss of information, in contrast to histograms and GLCM statistics, which only provide a synthetic representation of the precipitation field texture. In addition, using raw data exempts us from the complex and time-consuming process of feature extraction. On the other hand, classification algorithms have to be powerful enough to deal with high-dimensional inputs, especially when using high-resolution model outputs.

## 4 | TEXTURE DETECTION METHODS

This section introduces the three classifiers tested for the discrimination of rainfall texture: the similarity-based method, logistic regression, and the random forest method. These classifiers aim at estimating a categorical variable $Y$ with $K$ possible classes based on a vector of predictive variables $X \in \mathbb{R}^p$. In the present study, $Y$ is the texture type and $K=3$, with classes corresponding to no precipitation, intermittent rainfall and continuous rainfall. The predictors used are those described in section 3.

### 4.1 | The similarity-based method

The similarity-based method (hereafter denoted S) is directly inspired by the approach described by Raynaud et al. (2019). For texture characterisation, the similarity between the local precipitation distribution and the prescribed reference distributions for the three target classes (no/continuous/intermittent precipitation) is assessed by computing the $l_1$ distance between their respective cumulative distribution functions:

$$d_k = \frac{1}{B} \sum_{i=1}^{B} |F_{\mathrm{ref}}^{k}(i) - F(i)|, \quad k = 1, \dots, K,$$

where $F_{\mathrm{ref}}^k$ is the reference CDF for class $k$, $F$ is the local CDF and $B$ is the number of bins. The predicted class $k^*$ is the one that minimizes the distance $d_k$:

$$k^* = \arg \min_k \{d_k\}.$$

The reference CDFs used are those presented in Figure 1.

### 4.2 | Multinomial logistic regression

The multinomial logistic regression (hereafter denoted LR) model estimates the probability of each class according to a set of predictor variables using the following relation (Friedman et al., 2001):

$$\log \left( \frac{\mathbb{P}(Y = k | X)}{\mathbb{P}(Y = K | X)} \right) = \beta_{k,0} + \beta_k^T X, \quad k = 1, \dots, K - 1,$$

where, $\beta_{k,0}$ is the intercept and $\beta_k=(\beta_{k,1}, \dots, \beta_{k,p})$ are the regression coefficients for the class $k$. The class $K$ in the denominator is arbitrarily chosen to serve as the baseline class.
<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /><figcaption>FIGURE 2 Probability distributions of the GLCM statistics for the continuous (dark grey) and intermittent (light grey) rainfall textures, computed from the training dataset of 1-hour accumulated precipitation Arome forecasts with a pair of pixels at distance d= 1 pixel and averaged over the orientations θ∈[0°,180°,±45°,±90°,±135°]. The Kolmogorov–Smirnov distance between the two texture distributions is indicated at the top of each panel</figcaption></figure>

The resulting probabilities are then given by

$$ 
\begin{aligned}
\mathbb{P}(Y = k | X) &= \frac{e^{\beta_{k,0} + \beta_k^T X}}{1 + \sum_{l=1}^{K-1} e^{\beta_{l,0} + \beta_l^T X}}, & k = 1, \dots, K-1, \\
\mathbb{P}(Y = K | X) &= \frac{1}{1 + \sum_{l=1}^{K-1} e^{\beta_{l,0} + \beta_l^T X}}.
\end{aligned}
$$

The regression coefficients β are estimated by the method of maximum likelihood (James et al., 2014) from a training dataset of predictors and its associated ground-truth labels. In the prediction step, the predicted class k* of a new observation X is the one that maximizes P(Y = k|X):

$$ k^* = \arg \max_k \{ \mathbb{P}(Y = k | X) \} $$

This model has the advantage of being free of tunable parameters and the assumption of normality. It is implemented using the Python SCIKIT-LEARN package (Pedregosa et al., 2011).

## 4.3 | Random forests

Decision trees are built based on the Classification and Regression Trees (CART) algorithm (Breiman et al., 2017). It is a binary recursive algorithm that partitions the predictors space into homogeneous (or pure) regions (also known as tree nodes). A tree is built by splitting each internal node into two child nodes, using the predictor variable and threshold that maximize the improvement in node purity. Recursive splitting is then repeated until node purity is achieved (i.e., all the samples at that node have the same class label) or a stopping criterion is satisfied. Each terminal node (also called a leaf) is then assigned a class label, corresponding to the majority class of training samples in it. Once the decision tree has been generated from a training dataset, the class label of a new observation X is given by the class of the leaf node it falls in.

Decision trees have the advantage of mimicking the human decision-making process (James et al., 2014) and they are intuitive to understand. In addition, since they
make no assumption about the relationship between the predictor variables and the response variable, they allow for more complex predictive models than LR, which assumes a linear relationship through the logit function. The main drawback of decision trees is their prone-ness to overfitting, especially when a tree is deep. In that case, decision trees lose generalization capability on new data.

To alleviate this problem, ensemble methods such as random forests (hereafter denoted RFs: Breiman, 2001) can be used. The aim of RFs is to reduce the variance of the model by building multiple uncorrelated trees based on sample bagging and feature bagging. Each tree is built from a different training sample, drawn randomly from the original training database using a bootstrap procedure. In addition, node splitting is done by using a random subset of *m* predictors with $1 \le m < p$, where *p* is the total number of predictors. The resulting classification is then obtained by aggregating the prediction of each tree with a majority vote.

The RF algorithm has several hyperparameters that have to be tuned, in particular the number of variables used for node splitting, the number of trees, and the stopping criterion. RFs are implemented using the Python `SCIKIT-LEARN` package.

## 4.4 | Measure of feature importance

Interpretation of machine-learning algorithms is a key aspect in understanding how these models make predictions (McGovern *et al.*, 2019). Among the variety of model interpretation techniques, feature importance aims to identify the predictive variables that have the largest impact on model predictions.

For that purpose, the permutation importance method (also known as mean decrease accuracy or MDA for short: Breiman, 2001) measures variable importance from the impact on model accuracy of permuting each predictor variable randomly. A feature is then considered important if its permutation leads to a significant decrease in model performance. The permutation importance strategy presents several advantages. First, it does not require retraining the model: the prediction step is run on the permuted validation sample using the already-trained model. Secondly, it can be applied to any machine-learning model. Nonetheless, Strobl *et al.* (2008), Genuer *et al.* (2010), Díaz-Uriarte and De Andres (2006), and Gregorutti *et al.* (2017) point out that this method can be affected by the correlation of predictors.

# 5 | RESULTS

## 5.1 | Implementation of texture classifiers

A per-pixel texture classification needs predictive information derived from the close environment of each pixel. For that purpose, a common approach is to run a sliding-window classifier over the image that computes a single texture label for each window. In the present study, a km-wide circular patch is extracted around each grid point to capture its immediate neighbourhood. Patch size is a tuning parameter that determines the local context used to label the centre grid cell. Each patch is treated as an independent sample (however, the independence assumption is not strictly true, since patches are overlapping).

To reduce computational cost and because we are interested in texture characterisation at a predictable scale, the algorithm is run over a coarser latitude-longitude regular grid with a horizontal grid spacing of about 25 km. Within each neighbourhood, textural features are computed from the high-resolution field. The final texture classification is obtained with a nearest-neighbour spatial interpolation of the low-resolution Arome grid classification results.

The performance of the classifiers is evaluated using the Jaccard index, which is a standard metric in semantic segmentation. It is defined as the size of the intersection of the prediction and the ground truth over the size of their union. The Jaccard index can be used to assess the overall performance of the models as well as the class-specific performance (Appendix A). The statistical significance of the score difference between two models is assessed using the Kolmogorov-Smirnov test with a 5% risk.

## 5.2 | RF hyperparameter selection

As explained in section 4.3, random forests have three major parameters to tune: the number of predictive features used as split candidates, the number of trees, and their maximum depth. Their selection was optimized with a manual search by varying their values in the range {1, $\sqrt{p}$/2, $\sqrt{p}$, 2$\sqrt{p}$, 4$\sqrt{p}$}, {1,10,50,100,200,300,400,500}, and {1,5,10,15,20,25,30,35,40}, respectively. The parameter combination that minimizes the validation error is selected and presented in Table 1. Regarding the number of split candidates, $\sqrt{p}$ is found to provide the best results, which is consistent with the literature (Breiman, 2001; Friedman *et al.*, 2001; Genuer, 2010). One can also note that the number of trees and their depth tend to increase with the dimension of the feature vector. Hence, the RF model generated using the raw pixels as predictors
<figure><img src="image_8.png" /></figure>

<table><caption>**TABLE 1** Optimal values of random forest hyperparameters for each texture descriptor</caption><thead><tr><th></th><th>Raw pixels</th><th>CDF</th><th>GLCM</th></tr></thead><tbody><tr><td>Number of trees</td><td>200</td><td>100</td><td>50</td></tr><tr><td>Total number of predictors (p)</td><td>2,113</td><td>26</td><td>13</td></tr><tr><td>Number of split predictors</td><td>45</td><td>5</td><td>3</td></tr><tr><td>Tree depth</td><td>30</td><td>15</td><td>10</td></tr></tbody></table>

<table><caption>**TABLE 2** Optimal configuration for each descriptor–classifier combination. $D_n$ and $d$ denote the neighbourhood size and the pixel pair separation, respectively</caption><thead><tr><th></th><th>Raw pixels</th><th>CDF</th><th>GLCM</th></tr></thead><tbody><tr><td>S</td><td>×</td><td>$D_n$ = 120 km</td><td>×</td></tr><tr><td>LR</td><td>$D_n$ = 180 km</td><td>$D_n$ = 120 km</td><td>$D_n$ = 90 km<br>d = 1 pixel</td></tr><tr><td>RF</td><td>$D_n$ = 120 km</td><td>$D_n$ = 90 km</td><td>$D_n$ = 90 km<br>d = 1 pixel</td></tr></tbody></table>

Note: The total number of predictors and the number of split predictors are expressed in pixels, bins, and statistics for the raw pixels, CDF, and GLCM descriptors, respectively.

needs numerous and deeper trees, because of the high dimensionality of the data.

## 5.3 | Predictor tuning

For the three predictors introduced in section 3, their computation must be optimized to provide the most relevant textural information to the classifiers. This includes determining the neighbourhood diameter $D_n$ for the three predictors examined, the appropriate binning for the CDF and the GLCM, and the pixel pair distance $d$ for the GLCM. Similarly to the RF hyperparameter selection, these values have been optimized manually based on the validation error. The optimal configurations of the resulting seven classification models are presented in Table 2. The optimal neighbourhood size varies between 90 and 180 km, depending on both the predictor and the classifier, with larger neighbourhoods required when raw pixels are used. In addition, a fine discretization of the interval [0,10] mm is beneficial to all models using CDF and GLCM, as might be expected from Figure 1.

## 5.4 | Model performance evaluation

Figure 3 presents the Jaccard indices obtained with the seven models (hereafter denoted S, LR px, LR CDF, LR GLCM, RF px, RF CDF, RF GLCM) on the 80-case

validation set. Regarding continuous rainfall (Figure 3c), all models have an average Jaccard index between 0.5 and 0.6 and the LR px model is outperformed by all the other models. The Kolmogorov–Smirnov test indicates that the CDF-based machine-learning classifiers are not statistically different from each other, and the same holds true for the GLCM-based machine-learning classifiers (not shown). The cases of continuous precipitation for which the Jaccard index is zero correspond mainly to the misclassification of some intermittent-rainfall regions as continuous rainfall in images containing only intermittent rainfall. On the other hand, larger differences are observed for the detection of intermittent rainfall (Figure 3b), with an average Jaccard index between 0.2 and 0.6 depending on the classifier, consistent with the intuitive idea that intermittence may be more complex to characterise. In that case, the LR px method is outperformed by all the other models, and its performance is much lower than for continuous rainfall, with an average Jaccard index around 0.2. The performance of logistic regression appears to be strongly dependent on the predictor used, with the worst (respectively best) performance obtained with the raw pixels (respectively GLCM). Random forests provide the best classification results, whatever the predictor. The best classifiers (all RFs and LR GLCM) are also more robust, as reflected by the smaller ranges of Jaccard indices, and their performance is close to that obtained for the continuous texture. The global Jaccard index (Figure 3a) has similar behaviour, but values are larger because of the impact of nonprecipitating pixels. The Kolmogorov–Smirnov test indicates that RF px is significantly better than all other classifiers except RF GLCM.

Overall, these results provide relevant insight into the behaviour of each classifier for this texture classification problem. Interestingly, random forests are able to extract relevant information from both the CDF, which a priori contains only partial information about texture, and the high-dimensional and unprocessed information given by the raw pixels. In contrast, logistic regression needs a texture-oriented predictor to perform well. The good performance of RF with raw data is a very promising result, since this configuration has the advantage of requiring less feature engineering.

With average global Jaccard index values around 0.7, the performance of the best classifiers could seem low. First, it is important to mention that we are not interested in an exact per-pixel match between the prediction and the ground truth. Secondly, since there are no clear definitions of continuous and intermittent rainfall, the ground truth has been constructed based on expert interpretation. Given the subjective nature of these human labels, it is unrealistic to expect perfect model performance. One typical example of a tricky situation to label is a large
<figure><img src="image_9.png" /><figcaption>**FIGURE 3** Performance of the seven classification models, measured by (a) the average Jaccard index, (b) the intermittent rainfall class Jaccard index, and (c) the continuous rainfall class Jaccard index, over 80 validation cases (in the form of box plots); their mean value is indicated by the grey line. A grey star indicates that the score distribution of the classifier is statistically different from the score distribution of the similarity-based method according to the Kolmogorov-Smirnov test</figcaption></figure>

intense convective cell embedded in a wide convective area, as illustrated in Figure 4a. Depending on the expert, the predictable scales or the expectation of end users, it could be considered either as continuous rainfall or as part of a larger scale intermittent-rainfall area. Figures 4b,c show the ground-truth labels and the corresponding RF px classification. While the expert identifies two continuous patterns over Eastern France, the classifier identifies a large intermittent structure containing both large and small-scale cells. The uncertainty associated with this situation is also reflected by the diversity of the classification results obtained with the different models implemented (not shown). For instance, RF GLCM is mostly consistent with RF px, whereas RF CDF and LR GLCM are closer to the ground truth.

## 5.5 | Case studies

Since a visual inspection of the classification results is also an important criterion for their evaluation, this section examines the results obtained for two particular samples of the validation set. Figure 5 shows one of the best-performing examples (according to the global Jaccard index, equal to 0.83 for RF px). Consistently with the previous objective evaluation, all models are able to detect continuous precipitation areas accurately, while larger discrepancies are observed regarding the detection of intermittent regions. The similarity-based method provides only a partial detection of intermittent patterns. The dependence of LR performance on the predictor is also clearly noticeable. For instance, with raw pixels this classifier is not able to provide a large-scale texture signal. On the other hand, the three RF classification results are very close to the truth. The main difference between these three models is very subtle and it relates to the borders between continuous precipitation and nonprecipitating areas, which are wrongly considered as intermittent areas

when using the CDF. This behaviour may explain the slightly lower Jaccard index value observed in Figure 3. This example also illustrates how the information provided by the CDF can lead to different classification results, depending on the classifier.

Figure 6 shows one of the worst-performing examples (with a global Jaccard index of 0.53 for RF px). This case is very different from the previous one, since it contains only widely scattered light precipitation. Most intermittent patterns are properly detected by the three RFs and the LR GLCM classifiers (only results from RF px are shown for conciseness). Errors correspond mainly to an underestimation of rainfall areas, which does not seem problematic for practical applications. This suggests that low-Jaccard classifications from robust models still provide relevant information from a user point of view.

## 5.6 | Feature importance

As explained in section 4.4, measures of feature importance can be very useful for understanding the predictions of a classification model. In this section, importance measure results are presented for the CDF and GLCM-based models. The case of pixel-based models is not discussed, because the high dimensionality of the data makes the results difficult to interpret.

Figure 7 shows the importance of the CDF bins for LR and RF classifiers. As expected, the first bins are the most informative for both models. More precisely, the first bin, corresponding to the interval [0,0.1] mm, has the largest impact on the classification results. The RF and LR performance decreases exponentially with bin number up to bins 3 and 4, respectively. For higher bins, the decrease is small but negative, which means that these bins are probably useless for the predictions. It may also be noticed that the amplitude of the performance decrease is stronger with logistic regression. For instance, permuting the first bin
<figure><img src="image_10.png" /><figcaption>**FIGURE 4** (a) One-hour precipitation from the Arome model, valid on August 08, 2018 at 1800 UTC (in mm), (b) hand labels for continuous and intermittent rainfall in black and light grey, respectively, and (c) the classification obtained with RF px; the associated Global Jaccard index *J* is indicated in the legend [Colour figure can be viewed at wileyonlinelibrary.com][Colour figure can be viewed at wileyonlinelibrary.com]</figcaption></figure>

leads to a worsening of model performance of 52% for LR and 38% for RF. This result suggests that the RF classifier may be more robust to predictor changes.

Figure 8 presents the importance of the GLCM statistics for LR and RF classifiers. Noticeable differences

are observed between the behaviour of the two classifiers. Regarding logistic regression, the permutation of 10 out of 13 statistics leads to a significant performance decrease, reaching nearly 60% for the statistic $f_{13}$ (information measure of correlation). Regarding random forests, the most important statistic is $f_6$ (sum average), with a performance decrease of 25%, while the importance of the other statistics is much lower. Hence, the RF classifier may be more robust and may require fewer input features to perform well. It is also interesting to notice that the discrimination potential of the statistics $f_6$ and $f_{13}$ was already identified from the distributions in Figure 2.

## 5.7 | Feature selection

Importance measures can also be useful for variable selection, in order to reduce the number of predictors. A common approach is to extract a subset of variables according to the ranking given by the permutation importance (Genuer *et al.*, 2010).

Based on the results of Figure 7, a reduced RF CDF model has been designed by considering only the first three bins. Its performance is similar to that of the full model (according to the Kolmogorov-Smirnov test), which suggests that a fine binning of the CDF over the [0,1] mm interval may be sufficient to discriminate properly between rainfall textures with the random forests classifier. For LR CDF, a reduced model built by keeping only the first four bins (corresponding to the interval [0,2] mm) also achieves similar performance to the full model.

Regarding GLCM-based models, a subset of relevant predictors can be obtained by keeping statistics with positive importance. A subset of four statistics ($f_1, f_3, f_6, f_{13}$), hereafter called the interpretation subset, is obtained for the RF GLCM model. For the LR GLCM model, the interpretation subset is composed of 10 statistics ($f_1, f_3, f_5, f_6, f_8, f_9, f_{10}, f_{11}, f_{12}, f_{13}$). These subsets are then reduced further by taking into account the correlations of the statistics (by using a correlation threshold of 0.7) and their computational cost. The resulting subsets for RF GLCM and LR GLCM, hereafter called prediction subsets, are quite similar ($f_1, f_3, f_6$ and $f_1, f_3, f_6, f_{11}, f_{13}$, respectively). The statistics ($f_3, f_{13}$), ($f_1, f_6$), and $f_{11}$ characterise the correlation, the homogeneity, and the entropy of pixel intensities in the image, respectively.

The models generated with the full set, the interpretation set, and the prediction set achieve similar performance (according to the Kolmogorov-Smirnov test). It has also been verified visually that the reduced models lead to very similar classification results.
<figure><img src="image_11.png" /><figcaption>**FIGURE 5** (a) One-hour precipitation from the Arome model, valid on December 25, 2017 at 1200 UTC (in mm); hand labels for continuous and intermittent rainfall are overlaid in black and light grey, respectively. (b)–(h) Classification results obtained with the seven models. The global Jaccard index *J* of each classification is indicated in the legend [Colour figure can be viewed at wileyonlinelibrary.com]</figcaption></figure>

**6 | TRANSFER LEARNING**

Texture characterisation has been performed so far using training and validation datasets from Arome 1.3-km outputs interpolated on a 2.5-km grid. A similar classification may be performed on the range of available Arome precipitation outputs, in particular the Arome 1.3-km model interpolated on a 1-km grid and the Arome-EPS outputs (on a 2.5-km grid). The classification may also be applied to gridded observations for forecast verification. The Quantitative Precipitation Estimate (QPE) gridded product available at Météo-France is Antilope. Antilope merges spatial interpolation of rain-gauge data with radar-based rainfall data over the French territory. However, creating large hand-labelled training datasets for these new applications is time-consuming. In

order to avoid this step, one possibility is to apply transfer learning, which consists of reusing a model developed for one problem to solve a different problem (Pan and Yang, 2009). Hence, the aim of this section is to examine whether the models developed with and for Arome 2.5-km outputs can provide relevant results when applied to Arome 1 km, Arome-EPS, and the Antilope observations. For that purpose, a validation dataset has been constructed manually for each new NWP model considered and for the observations. Only the three full RF-based classifiers have been considered, since, according to previous results, they are the best candidates for operational applications. The application of the pretrained RF GLCM and RF px classifiers to the Arome 1-km outputs required some adjustments. The GLCM is now computed with a distance *d*=3 pixels (instead of 1), so that the kilometre distances used
<figure><img src="image_12.png" /></figure>

<figure><img src="image_13.png" /><figcaption>**FIGURE 6** (a) One-hour precipitation from the Arome model, valid on January 11, 2019 at 2400 UTC (in mm); hand labels for continuous and intermittent rainfall are overlaid in black and light grey, respectively, and (b) classification result obtained with the RF px model; the associated global Jaccard index *J* is indicated in the legend. The total counts of pixels classified as intermittent rain are 200,338 and 131,014 in the Ground Truth and RF px classifications, respectively [Colour figure can be viewed at wileyonlinelibrary.com]</figcaption></figure>

for training and validation are roughly the same. For the pixel-based RF, a subsampling of the neighbourhood pixels (which consists of keeping only one pixel over two or three in both directions) is necessary to preserve the dimension of the training predictor vector. Figure 9a–d illustrates the differences that can be expected between the four datasets, for a case that contains a large variety of rainfall textures. Arome outputs on the 2.5- and 1-km grids are very similar, since they are both obtained from the 1.3-km Arome model, and they provide finer-scale details than the 2.5-km Arome-EPS predictions. Despite these differences, all simulations provide quite a realistic description of the observed situation.

The three RF classifiers trained on Arome 2.5-km outputs perform well when applied to Arome-EPS outputs. They achieve a similar performance, in term of Jaccard index, to the one obtained for Arome 2.5-km classifications. From a subjective point of view, the examination of Figure 9 (third column) also indicates relevant classifications of both continuous and intermittent rainfall. This suggests that the difference in horizontal grid spacing between the deterministic and ensemble NWP models has no significant impact on texture classification, at least when precipitation forecasts are postprocessed on the same grid.

The average objective performances of the RF CDF and RF GLCM classifiers on the Arome 1-km validation dataset are very close to those obtained for Arome 2.5-km classifications and they clearly outperform the RF px performance. These results are also supported by the example of Figure 9 (second column).

Understanding the lower performance of the RF px model is complicated, since we have not investigated deeply how this model operates (i.e., what

it learns exactly), but it may be affected by the subsampling step.

However, a simple and straightforward solution to improve the performance of the RF px classifier is to retrain this statistical model with a cheap training set constructed by simply interpolating the Arome 2.5-km hand labels on the 1-km grid. This retrained model improves the objective and subjective classification performance significantly (Figure 10).

Lower performance is observed, on average, for the texture classification of Antilope precipitation fields with the three RF classifiers. The example presented in the last column of Figure 9 summarises the main classification characteristics observed over the whole database well. Intermittent areas are generally properly detected. On the other hand, the detection accuracy for continuous rainfall areas depends on their size: large regions are mostly well identified, while smaller regions are often misclassified as intermittent. A detailed examination of the ground-truth database has revealed an adaptation of the expert labelling rules to the reduced Antilope domain, leading to a segmentation of smaller precipitating cells. For instance, medium-size convective cells embedded in a wide convective area are mostly labelled as continuous in the Antilope database, while they are more often classified as intermittent by the RF models. These new labelling rules are questionable, since the same scales need to be detected in the prediction and the observations for proper verification. Because of these labelling flaws, our evaluation is based mainly on a visual inspection of the validation cases. The results of Figure 9 indicate that the three RF classifiers give an acceptable classification that could provide an interesting verification information.
<figure><img src="image_14.png" /><figcaption>(a) LR CDF</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>(b) RF CDF</figcaption></figure>

**FIGURE 7** Importance of CDF bins, as measured by the decrease of average Jaccard index after feature permutation in the validation set. (a) Logistic regression and (b) random forests. The box plots represent results obtained over 100 runs of the permutation importance algorithm. The bins are sorted according to their importance. The dashed line represents the importance threshold value corresponding to a 0% decrease in accuracy

Finally, it is worth mentioning that transfer learning could also be seen as a cost-effective method to design new training databases or increase existing ones, by providing first-guess hand labels that could then be modified manually.

## 7 | CONCLUSION AND FUTURE WORK

This article evaluated the performance of several classification methods in detecting continuous and intermittent rainfall patterns in kilometer-scale Arome forecasts. For that purpose, the reference similarity-based method, which has been applied successfully to detect intensity-based rainfall objects, has been compared with more advanced classifiers from the machine-learning domain, and textural features of increasing complexity have been tested. Training and validation for the different models were based on a unique hand-labelled database constructed from several hundred deterministic Arome forecasts.

<figure><img src="image_16.png" /><figcaption>(a) LR GLCM</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>(b) RF GLCM</figcaption></figure>

**FIGURE 8** Importance of GLCM statistics, as measured by the decrease of average Jaccard index after feature permutation in the validation set. (a) Logistic regression and (b) random forests. The box plots represent results obtained over 100 runs of the permutation importance algorithm. The features are sorted according to their importance. The dashed line represents the importance threshold value corresponding to a 0% decrease in accuracy

While continuous rainfall is identified properly by all models, the accuracy of intermittent rainfall detection clearly depends on the classifier and the predictors used. The similarity-based method, which only provides a partial detection of these areas, is outperformed by almost all machine-learning algorithms. The performance of logistic regression is improved with a texture-oriented predictor such as the GLCM statistics. On the other hand, random forests provide the best classification results whatever the predictor, with a surprising ability to extract relevant information from a synthetic descriptor such as the CDF and from the raw neighbouring pixels. Due to the subjectivity of human rainfall texture labels, a visual examination of the resulting discrimination shows generally more promise than indicated by the objective scores.

The capacity of the trained random forest models to generalise on precipitation outputs from different versions of the Arome model (including ensemble outputs and different postprocessing grids) and on observations was finally examined. This transfer of knowledge is shown to work quite well in most cases, especially with the CDF and GLCM-based models.
<figure><img src="image_18.png" /><figcaption>**FIGURE 9** One-hour accumulated precipitation forecasts from (a) Arome 2.5 km, (b) Arome 1 km, (c) one member of Arome-EPS, and (d) corresponding observations, valid on June 03, 2018 at 1800 UTC. The observation domain used in (d) is indicated with bounding boxes in panels (a)–(c). (e)–(h) The corresponding ground-truth texture labels and the corresponding classification results obtained with (i)–(l) RF px, (m)–(p) RF CDF, and (q)–(t) RF GLCM. The global Jaccard index *J* of each classification is indicated in the legend. All classifiers are trained on the original Arome 2.5-km database [Colour figure can be viewed at wileyonlinelibrary.com]</figcaption></figure>

In a real-time application, GLCM-based classification models are the most costly, due to the computational cost of the associated statistics (even with a reduced set). Random forests using either raw pixels or CDF may then be preferred, since they achieve similar performance at a lower cost. The numerical efficiency could be increased further using gradient boosting. Boosting is another tree-based ensemble classifier, which applies a set of weights to the training dataset during the learning process

in order to put greater emphasis on misclassified samples (Freund et al., 1996). Preliminary results indicate that gradient boosting achieves similar performance to random forests, with the advantage of requiring shallower trees (and fewer trees with CDF and raw pixels). The reduction of the prediction-phase cost compared with RFs is around 60%.

An analysis of feature importance allowed for an explanation of black-box machine-learning methods for the
<figure><img src="image_19.png" /><figcaption>**FIGURE 10** Classification obtained with the retrained RF px model for the example of Figure 9b. The corresponding global Jaccard index is equal to 0.71</figcaption></figure>

CDF and GLCM predictors. It would also be interesting to understand better what the pixel-based models learn. This could be achieved with pixel activation maps to highlight selected pixels as a function of their minimal depth distribution, for instance (Ishwaran *et al.*, 2010).

The extension of these classification methods to other fields, such as cloud cover, the texture of which is close to the precipitation one, will be investigated in a future work. The object-based processing of high-resolution forecasts is a promising approach for the verification of NWP models and for their utilisation by forecasters and other end users. Hence, future studies will focus on defining and detecting high-impact weather objects. In addition to intensity and texture criteria, this detection could be based on shape criteria (e.g., to identify V-shaped thunderstorms, bow echoes) and multivariate inputs. The benefit of using convolutional neural networks to handle these more complex aspects will be addressed.

ORCID

Yamina Hamidi https://orcid.org/0000-0003-4007-6678

Laure Raynaud https://orcid.org/0000-0003-1656-9982

REFERENCES

Arbogast, P., Pannekoucke, O., Raynaud, L., Lalanne, R. and Mémin, E. (2016) Object-oriented processing of CRM precipitation forecasts by stochastic filtering. *Quarterly Journal of the Royal Meteorological Society*, 142, 2827–2838

Bankman, I. (2008) *Handbook of Medical Image Processing and Analysis*. Johns Hopkins University, Baltimore, MD: Elsevier.

Bouttier, F., Vié, B., Nuissier, O. and Raynaud, L. (2012) Impact of stochastic physics in a convection-permitting ensemble. *Monthly Weather Review*, 140(11), 3706–3721

Breiman, L. (2001) Random forests. *Machine Learning*, 45(1), 5–32

Breiman, L., Friedman, J., Olshen, R. and Stone, C. (2017). *Classification and regression trees*, (Routledge).

Brousseau, P., Seity, Y., Ricard, D. and Léger, J. (2016) Improvement of the forecast of convective activity from the Arome-France system. *Quarterly Journal of the Royal Meteorological Society*, 142(699), 2231–2243

Chen, C. H., Pau, L. F. & Wang, P. S. P. (1993) *Handbook of pattern recognition and computer vision*. Singapore: World Scientific Publishing Co. Pte. Ltd. 395–441.

Clark, A.J., MacKenzie, A., McGovern, A., Lakshmanan, V. and Brown, R.A. (2015) An automated, multiparameter dryline identification algorithm. *Weather and Forecasting*, 30(6), 1781–1794

Davis, C., Brown, B. and Bullock, R. (2006) Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas. *Monthly Weather Review*, 134(7), 1772–1784

Descamps, L., Labadie, C., Joly, A., Bazile, E., Arbogast, P. and Cébron, P. (2015) PEARP, the Météo-France short-range ensemble prediction system. *Quarterly Journal of the Royal Meteorological Society*, 141(690), 1671–1685

Díaz-Uriarte, R. and De Andres, S.A. (2006) Gene selection and classification of microarray data using random forest. *BMC Bioinformatics*, 7(1), 3

Drönner, J., Korfhage, N., Egli, S., Mühling, M., Thies, B., Bendix, J., Freisleben, B. and Seeger, B. (2018) Fast cloud segmentation using convolutional neural networks. *Remote Sensing*, 10(11), 1782

Ebert, E. and McBride, J. (2000) Verification of precipitation in weather systems: determination of systematic errors. *Journal of Hydrology*, 239(1-4), 179–202

Ebert, E.E. (2008) Fuzzy verification of high-resolution gridded forecasts: a review and proposed framework. *Meteorological Applications*, 15(1), 51–64

Freund, Y., Schapire, R.E. and Robert E. (1996). Experiments with a new boosting algorithm, *ICML*, 148–156.

Friedman, J., Hastie, T. and Tibshirani, R. (2001) *The Elements of Statistical Learning* Vol. 1. New York: Springer. Springer Series in Statistics.

Gandomi, A.H., Alavi, A.H. and Ryan, C. (2015) *Handbook of Genetic Programming Applications*. Switzerland: Springer International Publishing.

Genuer, R. (2010) *Forêts aléatoires: aspects théoriques, sélection de variables et applications*. Paris XI: Université Paris Sud. PhD Thesis.

Genuer, R., Poggi, J.-M. and Tuleau-Malot, C. (2010) Variable selection using random forests. *Pattern Recognition Letters*, 31(14), 2225–2236

Gilleland, E., Ahijevych, D., Brown, B., Casati, B. and Ebert, E. (2009) Intercomparison of spatial forecast verification methods. *Weather and Forecasting*, 24, 1416–1430

Gregorutti, B., Michel, B. and Saint-Pierre, P. (2017) Correlation and variable importance in random forests. *Statistics and Computing*, 27(3), 659–678

Hall-Beyer, M. (2017) Practical guidelines for choosing GLCM textures to use in landscape classification tasks over a range of moderate spatial scales. *International Journal of Remote Sensing*, 38(5), 1312–1338

Haralick, R.M., Shanmugam, K. and Dinstein Its' Hak (1973) Textural features for image classification. *IEEE Transactions on systems, man, and cybernetics*, 6, 610–621
Ishwaran, H., Kogalur, U.B., Gorodeski, E.Z., Minn, A.J. and Lauer, M.S. (2010) High-dimensional variable selection for survival data. *Journal of the American Statistical Association*, 105(489), 205–217

James, G., Witten, D., Hastie, T. and Tibshirani, R. (2014) *An Introduction to Statistical Learning: With Applications in R*. New York: Springer Publishing Company.

Lowitz, G.E. (1983) Can a local histogram really map texture information?. *Pattern Recognition*, 16(2), 141–147

Mallat, S.G. (1989) A theory for multiresolution signal decomposition: the wavelet representation. *IEEE Transactions on Pattern Analysis and Machine Intelligence*, 7, 674–693

Marzban, C. and Sandgathe, S. (2008) Cluster analysis for object-oriented verification of fields: a variation. *Monthly Weather Review*, 136(3), 1013–1025

Materka, A. and Strzelecki, M. (1998) *Texture Analysis Methods–A Review*, (pp. 9–11). Brussels: Technical University of Lodz, Institute of Electronics. COST B11 report.

McGovern, A., Lagerquist, R., Gagne, D.J., Jergensen, G.E., Elmore, K.L., Homeyer, C.R. and Smith, T. (2019) Making the black-box more transparent: understanding the physical implications of machine learning. *Bulletin of the American Meteorological Society*, 100(11), 2175–2199

Pan, S.J. and Yang, Q. (2009) A survey on transfer learning. *IEEE Transactions on Knowledge and Data Engineering*, 22(10), 1345–1359

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M. and Duchesnay, E. (2011) Scikit-learn: Machine learning in python. *Journal of Machine Learning Research*, 12(Oct), 2825–2830

Raynaud, L. and Bouttier, F. (2016) Comparison of initial perturbation methods for ensemble prediction at convective scale. *Quarterly Journal of the Royal Meteorological Society*, 142(695), 854–866

Raynaud, L., I. Pechin, P. Arbogast, L. Rottner and M. Destouches (2019) Object-based verification metrics applied to the evaluation and weighting of convective-scale precipitation forecasts. *Quarterly Journal of the Royal Meteorological Society*, 145(722), 1992–2008. https://doi.org/10.1002/qj.3540. ISSN 0035-9009

Rodriguez-Galiano, V., Chica-Olmo, M., Abarca-Hernandez, F., Atkinson, P.M. and Jeganathan, C. (2012) Random forest classification of Mediterranean land cover using multi-seasonal imagery and multi-seasonal texture. *Remote Sensing of Environment*, 121, 93–107

Rottner, L., Arbogast, P., Destouches, M., Hamidi, Y. and Raynaud, L. (2019) The similarity-based method: a new object detection method for deterministic and ensemble weather forecasts. *Advances in Science and Research*, 16, 209–213

Seity, Y., Brousseau, P., Malardel, S., Hello, G., Bénard, P., Bouttier, F., Lac, C. and Masson, V. (2011) The Arome-France convective-scale operational model. *Monthly Weather Review*, 139(3), 976–991

Strobl, C., Boulesteix, A.-L., Kneib, T., Augustin, T. and Zeileis, A. (2008) Conditional variable importance for random forests. *BMC Bioinformatics*, 9(1), 307

Tuceryan, M. and Jain, A.K. (1993). Texture analysis., *Handbook of Pattern Recognition and Computer Vision*, pp. 235–276. World Scientific Publishing Co. Pte. Ltd.

Turner, M.R. (1986) Texture discrimination by Gabor functions. *Biological Cybernetics*, 55(2-3), 71–82

Welch, R., Sengupta, S. and Chen, D. (1988) Cloud field classification based upon high spatial resolution textural features: 1. Gray level co-occurrence matrix approach. *Journal of Geophysical Research*, 93, 12663–12681

Yousefi, S. and Kehtarnavaz, N. (2011). A new stochastic image model based on Markov random fields and its application to texture modeling. In:., *2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*, pp. 1285–1288: IEEE. https://doi.org/10.1109/ICASSP.2011.5946646.

**How to cite this article:** Hamidi Y, Raynaud L, Rottner L, Arbogast P. Texture-based classification of high-resolution precipitation forecasts with machine-learning methods. *QJR Meteorol. Soc.* 2020;146:3014–3028. https://doi.org/10.1002/qj.3823

## APPENDIX A

In a three-class classification problem, the confusion matrix (also known as contingency matrix) is given as shown in Table A1.

<table><caption>TABLE A1 Three-class contingency table</caption><thead><tr><th rowspan="2">Prediction</th><th colspan="3">Ground Truth</th></tr><tr><th>A</th><th>B</th><th>C</th></tr></thead><tbody><tr><th>A</th><td>AA</td><td>AB</td><td>AC</td></tr><tr><th>B</th><td>BA</td><td>BB</td><td>BC</td></tr><tr><th>C</th><td>CA</td><td>CB</td><td>CC</td></tr></tbody></table>

The Jaccard indices corresponding to the model's ability to predict classes A, B, and C are given by

$$J_A = \frac{AA}{AA + AB + AC + BA + CA}$$

$$J_B = \frac{BB}{AB + BB + CB + BA + BC}$$

$$J_C = \frac{CC}{CA + CB + CC + AC + BC}$$

The global Jaccard index averaged over the three classes is then given by

$$J = \frac{1}{3}(J_A + J_B + J_C).$$

Global and class-specific Jaccard indices are between 0 and 1; a value of 1 indicates a perfect match between the prediction and the ground truth.