# Improving TIGGE Precipitation Forecasts Using an SVR Ensemble Approach in the Huaihe River Basin

Authors: Chenkai Cai; Jianqun Wang; Zhijia Li
Journal: Advances in Meteorology
 (2018)

---

## Abstract

Recently, the use of the numerical rainfall forecast has become a common approach to improve the lead time of streamflow forecasts for flood control and reservoir regulation. The control forecasts of five operational global prediction systems from different centers were evaluated against the observed data by a series of area-weighted verification and classification metrics during May to September 2015–2017 in six subcatchments of the Xixian Catchment in the Huaihe River Basin. According to the demand of flood control safety, four different ensemble methods were adopted to reduce the forecast errors of the datasets, especially the errors of missing alarm (MA), which may be detrimental to reservoir regulation and flood control. The results indicate that the raw forecast datasets have large missing alarm errors (MEs) and cannot be directly applied to the extension of flood forecasting lead time. Although the ensemble methods can improve the performance of rainfall forecasts, the missing alarm error is still large, leading to a huge hazard in flood control. To improve the lead time of the flood forecast, as well as avert the risk from rainfall prediction, a new ensemble method was proposed on the basis of support vector regression (SVR). Compared to the other methods, the new method has a better ability in reducing the ME of the forecasts. More specifically, with the use of the new method, the lead time of flood forecasts can be prolonged to at least 3 d without great risk in flood control, which corresponds to the aim of flood prevention and disaster reduction.


---

## Body

## 1. Introduction

With rapid development of the economy and population growth, as well as the impact of climate change and increasing demand for freshwater, many regions of China are facing a shortage of water resources. At the same time, due to the influence of monsoon climate, flood disasters frequently occur in China during the rainy season, causing enormous economic losses to society [1]. Streamflow forecasting and flood regulation play a key role in China’s flood control and water resource management system [2–4]. Traditionally, the flood forecast approach takes precipitation from ground sites as its input; its forecast lead time is limited and cannot satisfy the demand for flood prevention and water resource utilization [5, 6]. Therefore, it is a most urgent and important task to make efforts to lengthen the lead time and improve the accuracy of flood forecasting. As many previous studies have mentioned, numerical rainfall forecasting is an effective way to solve this problem [7, 8].Recently, the revolution of computer technology and the progress of meteorological and climate models have brought about the continuous development of numerical weather forecasts [9–11]. As a major component of The Observing System Research and Predictability Experiment (THORPEX), TIGGE (the THORPEX Interactive Grand Global Ensemble) dataset consists of ensemble forecast datasets from eleven main forecasting centers worldwide, starting in 2006, aimed at improving the accuracy of high-impact weather forecasts within two weeks [12, 13]. A number of studies on TIGGE precipitation forecasts have been extensively carried out for extreme rain events and hydrological forecasting. Clark and Hey [14] applied the medium range numerical weather prediction model output from NCEP (National Centers for Environmental Prediction) to streamflow forecasting in the United States, but the results showed that the model apparently has low skill in predicting precipitation and temperature. Pappenberger et al. [15] improved the lead time of flood forecasts to 8 days in advance by using TIGGE data as a meteorological input to the European Flood Alert System. Additionally, another case in the Upper Huaihe Basin also showed that a reliable flood warning is available as early as 10 days in advance [16]. He et al. [17] found that the uncertainties in precipitation would dominate and propagate through a coupled atmospheric-hydrologic-hydraulic cascade system. Su et al. [18] evaluated the errors of quantitative precipitation forecasts and probabilistic quantitative precipitation forecasts from six operational global ensemble prediction systems in TIGGE during June to August 2008–2012 in the Northern Hemisphere. Sagar et al. [19] assessed the skill of a numerical weather prediction model for rainstorms over India.Due to the low skill of precipitation forecasts, many researchers have suggested that it is possible to improve the accuracy of the forecasts through downscaling and ensemble forecasting [20–24]. Although several ensemble methods have been used to reduce the error of rainfall forecasts, most of these methods established the relationship between different forecasts and the observed value to reduce the overall errors such as mean absolute errors (MAEs) or root mean square errors (RMSEs) which neglected the different effects of false alarm (FA, event forecasted to occur but did not occur) and missing alarm (MA, event forecasted not to occur but did occur) on flood control security [23, 25]. A FA only reduces the benefits of flood resources, while an MA is disadvantageous to flood control safety. As safety is the most important target in flood control, it is a main task to find a method which can provide reliable precipitation forecast to improve the lead time of flood forecast without bringing huge flood risk by large MEs (missing alarm errors).In this paper, we evaluated the predictions of five forecasting centers in the Huaihe River Basin of China during May to September 2015–2017. Both linear and nonlinear system analysis methods were applied to ensemble forecasting with the purpose of reducing the prediction errors. Furthermore, since flood control safety is the first priority in water resource management, specific attention will be paid to the errors of MAs to avoid increasing the risk in flood control; a new objective function will also be proposed. In the next section, we detail the evaluation indicators along with the correction methods, which include both linear and nonlinear methods. Section3 introduces an overview of the datasets and the study area. The results and discussion are described in Section 4, and finally, the conclusions are provided in Section 5.


## 2. Methodology

The main assumption in post-processing the forecasts is that the observation and forecast are correlated, and the future behavior of the system will remain the same. The purpose of this study is to lessen the ME of rainfall forecasts by means of system analysis so that rainfall prediction can be better utilized in flood forecasting without bringing about huge risk in flood control.


### 2.1. Verification Methods

The accuracy of precipitation prediction is usually used as an evaluation criterion to determine whether rainfall prediction information is available. To estimate the skill of the forecasts with different lead times, multiple classic statistical characteristics were assessed to compare the forecasts and observations, including MAE and RMSE. Meanwhile, a qualitative analysis was also adopted to complement the statistical characteristics based on the classification standard for daily rainfall formulated by the meteorological department of China. According to the total amount of rainfall in 24 h, this standard divides the daily precipitation into seven types: no rain, light rain, medium rain, heavy rain, rainstorm, heavy rainstorm, and extreme rainstorm. Generally, for meteorological researchers, no rain means that the daily precipitation is 0; however, due to the limited influence of rainfall of less than 1 mm on the formation of floods, the no rain standard was changed to less than 1 mm in this study, and each classification standard is detailed in Table1.Table 1
Classification standard of daily precipitation.
Magnitude
Classification standard of precipitation
Amount of daily precipitation (mm)
1
No rain
0–0.9
2
Light rain
1.0–9.9
3
Medium rain
10.0–24.9
4
Heavy rain
25.0–49.9
5
Rainstorm
50.0–99.9
6
Heavy rainstorm
100.0–249.9
7
Extreme rainstorm
>250.0On the basis of the magnitudes of forecasts and observations, three combinations of the results are as follows:Hit: the magnitude of the forecast is the same as that of the observationMiss alarm: the magnitude of the forecast is smaller than that of the observationFalse alarm: the magnitude of the forecast is larger than that of the observationThe skill of the forecasts can be assessed based on a contingency table that contains the frequency of the combinations. From the contingency table, the rates of hit, MA and FA can be computed for each magnitude:Hit rate (HR):(1)αi=ni,jNi×100%,i=j.MA rate (MAR):(2)βi=ni,jNi×100%,i<j.FA rate (FAR):(3)γi=ni,jNi×100%,i>j,where ni,j is the number of cases with the predicted magnitude of i and the measured magnitude of j; Ni is the number of the forecast for the magnitude of i.


### 2.2. Ensemble Methods

Due to the determinacy and randomness of the atmospheric motion, it is impossible to obtain the initial field of numerical prediction objectively and accurately, and a perfect weather prediction model does not exist [26]. Ensemble forecasting is an efficient method to eliminate the uncertainty caused by observation error and analysis error by using data from different sources [27, 28]. Several cases have proven that it is possible to improve the accuracy of precipitation forecasts by using ensemble methods. In former studies, the researchers usually used perturbed forecasts from a single dataset or control forecasts from several datasets as the input of ensemble forecast [29–31].In this paper, we used the control forecast of five selected datasets and compared three kinds of linear ensemble forecasting methods, namely, the ensemble mean (EM), bias-removed ensemble mean (BREM), and linear regression (LR) methods [32, 33]. Meanwhile, since the previous studies showed that the prediction errors of different magnitudes exhibit a nonlinear variation, support vector machine, as an effective nonlinear regression method, was also employed for ensemble forecasting.


#### 2.2.1. Bias-Removed Ensemble Mean

The bias-removed ensemble mean is developed from the ensemble mean, and the calculation formula is as follows:(4)F=O¯+1m∑i=1mFi−F¯i,where O¯ represents the average value of observations during the training period, m represents the number of members used in the ensemble forecast, Fi represents the forecast value of member i, F¯i represents the average value of member i during the training period, and F is the ensemble forecast value.


#### 2.2.2. Support Vector Regression

In formula (4), the relationship between the ensemble forecast value F and the forecast value of member F1, F2,…, Fm is linear. The relationship can be considered to be nonlinear as follows:(5)F=fF1,F2,…,Fm.In formula (5), f represents the nonlinear relationship. A BP artificial neural network model or a support vector regression model can be used to approximate f.Support vector machine (SVM) is a mechanical learning method for classification and regression proposed by Vanpik et al. in 1995 based on the structural risk minimization (SRM) principle [34]. SVR (support vector regression), with good generalization and nonlinear processing ability, has been widely employed in hydrology [35–38]. The methodology of SVR is briefly described below.The basic idea of SVR for nonlinear case is to map the original problem to a linear problem in a high-dimensional feature space by nonlinear transformation to approximatef with input vector x∈Rn, output vector y∈R and sample set x1,y1,…,xl,yl:(6)fx=w,ϕx+b.In formula (6), f(x) is the regression function, ϕx is the nonlinear transformation function, w is the weight vector, and b is the threshold. Then, the ε-SVR model is built by solving the optimization problem:(7)minw,ξi,ξi∗,b12wTw+C1l∑i=1lξi+ξi∗,s.t.w,ϕxi+b−yi≤ε+ξi,yi−w,ϕxi+b≤ε+ξi∗,ξi,ξi∗≥0.,i=1,2,…,l.The dual form of the problem can be expressed as(8)maxα,α∗∑i=1lαiyi−ε−αi∗yi+ε−12∑i=1l∑j=1lαi−αi∗αj−αj∗Kxi⋅xj,s.t.∑i=1lαi−αi∗=0,0≤αi,αi∗≤C,i=1,…,l,where, Kxi,xj=ϕxi,ϕxj represents the kernel function. After the Lagrange multipliers, αi and αi∗, in formula (8) have been determined, the ε-SVR regression function f(x) can be established as(9)fx=∑SVsα¯i−α¯i∗Kxi,x+b.In this study, the radial basis function kernel with a parameterσ was selected for nonlinear transformation:(10)Kxi,x=exp−x−xi22σ2.The parameterε in formula (8) should be determined beforehand, but in many practical cases, it is hard to determine the value of ε before training. In order to solve this problem, a new parameter ν was introduced by Schölkopf (ν-SVR) [39]. Subsequently, the optimization problem can be changed as(11)minw,b,ξ,ξ∗ε12wTw+Cνε+1l∑i=1lξi+ξi∗,s.t.w,ϕxi+b−yi≤ε+ξi,yi−w,ϕxi+b≤ε+ξi∗,ξi,ξi∗>0,i=1,…,l,ε>0.Also, the dual form of formula (11) is(12)maxα,α∗∑i=1lαiyi−ε−αi∗yi+ε−12∑i=1l∑j=1lαi−αi∗αj−αj∗Kxi⋅xj,s.t.∑i=1lαi−αi∗=0,∑i=1lαi+αi∗≤C⋅ν,0≤αi,αi∗≤Cl,i=1,…,l.The capability ofν-SVR depends on the parameter C, ν, and σ. The cost constant C is a compromise between the complexity and generalization of the model. It is used to adjust the ratio of confidence range and empirical risk in the sample space, which determines the penalty degree for the sample whose loss is greater than ε. The parameter ν represents the lower bound of support vector and the upper bound of gap error. Additionally, since ν is introduced to optimize ε which controls the width of insensitive band and affects the number of support vectors, ν also indirectly affects the number of support vectors and prediction accuracy, and the reduction of ν leads to the reduction of support vectors. Accordingly, the quadratic programming problem (formula (12)) can be solved by determining the three parameters and the MAE of the ensemble forecast can be calculated by the following formula:(13)MAE=1l∑i=1lfxi−yi.In formula (13), f(xi) and yi indicate the ith ensemble forecast and observed values, and l is the sample size.Usually, in order to get the best approximation ability, nonlinear optimization algorithm is used to optimize the parametersC, ν, and σ by minimizing the MAE. However, the number of support vectors can be very large if the ν-SVR model is solved by optimization algorithm. Therefore, the range of parameter ν should be analyzed with determined C and σ by perturbation method.In this paper, theν-SVR model was used to approximate f and the parameters of SVR were optimized by the PSO (particle swarm optimization) algorithm with the objective function of the minimization of the MAE [40, 41]. Furthermore, the number of support vectors was controlled to be less than l/2 by controlling the range of ν.All the SVR models used in this research are based on the open source software LIBSVM developed by Lin Chih-Jen. More information about the model is available athttps://www.csie.ntu.edu.tw/∼cjlin/.


## 2.1. Verification Methods

The accuracy of precipitation prediction is usually used as an evaluation criterion to determine whether rainfall prediction information is available. To estimate the skill of the forecasts with different lead times, multiple classic statistical characteristics were assessed to compare the forecasts and observations, including MAE and RMSE. Meanwhile, a qualitative analysis was also adopted to complement the statistical characteristics based on the classification standard for daily rainfall formulated by the meteorological department of China. According to the total amount of rainfall in 24 h, this standard divides the daily precipitation into seven types: no rain, light rain, medium rain, heavy rain, rainstorm, heavy rainstorm, and extreme rainstorm. Generally, for meteorological researchers, no rain means that the daily precipitation is 0; however, due to the limited influence of rainfall of less than 1 mm on the formation of floods, the no rain standard was changed to less than 1 mm in this study, and each classification standard is detailed in Table1.Table 1
Classification standard of daily precipitation.
Magnitude
Classification standard of precipitation
Amount of daily precipitation (mm)
1
No rain
0–0.9
2
Light rain
1.0–9.9
3
Medium rain
10.0–24.9
4
Heavy rain
25.0–49.9
5
Rainstorm
50.0–99.9
6
Heavy rainstorm
100.0–249.9
7
Extreme rainstorm
>250.0On the basis of the magnitudes of forecasts and observations, three combinations of the results are as follows:Hit: the magnitude of the forecast is the same as that of the observationMiss alarm: the magnitude of the forecast is smaller than that of the observationFalse alarm: the magnitude of the forecast is larger than that of the observationThe skill of the forecasts can be assessed based on a contingency table that contains the frequency of the combinations. From the contingency table, the rates of hit, MA and FA can be computed for each magnitude:Hit rate (HR):(1)αi=ni,jNi×100%,i=j.MA rate (MAR):(2)βi=ni,jNi×100%,i<j.FA rate (FAR):(3)γi=ni,jNi×100%,i>j,where ni,j is the number of cases with the predicted magnitude of i and the measured magnitude of j; Ni is the number of the forecast for the magnitude of i.


## 2.2. Ensemble Methods

Due to the determinacy and randomness of the atmospheric motion, it is impossible to obtain the initial field of numerical prediction objectively and accurately, and a perfect weather prediction model does not exist [26]. Ensemble forecasting is an efficient method to eliminate the uncertainty caused by observation error and analysis error by using data from different sources [27, 28]. Several cases have proven that it is possible to improve the accuracy of precipitation forecasts by using ensemble methods. In former studies, the researchers usually used perturbed forecasts from a single dataset or control forecasts from several datasets as the input of ensemble forecast [29–31].In this paper, we used the control forecast of five selected datasets and compared three kinds of linear ensemble forecasting methods, namely, the ensemble mean (EM), bias-removed ensemble mean (BREM), and linear regression (LR) methods [32, 33]. Meanwhile, since the previous studies showed that the prediction errors of different magnitudes exhibit a nonlinear variation, support vector machine, as an effective nonlinear regression method, was also employed for ensemble forecasting.


### 2.2.1. Bias-Removed Ensemble Mean

The bias-removed ensemble mean is developed from the ensemble mean, and the calculation formula is as follows:(4)F=O¯+1m∑i=1mFi−F¯i,where O¯ represents the average value of observations during the training period, m represents the number of members used in the ensemble forecast, Fi represents the forecast value of member i, F¯i represents the average value of member i during the training period, and F is the ensemble forecast value.


### 2.2.2. Support Vector Regression

In formula (4), the relationship between the ensemble forecast value F and the forecast value of member F1, F2,…, Fm is linear. The relationship can be considered to be nonlinear as follows:(5)F=fF1,F2,…,Fm.In formula (5), f represents the nonlinear relationship. A BP artificial neural network model or a support vector regression model can be used to approximate f.Support vector machine (SVM) is a mechanical learning method for classification and regression proposed by Vanpik et al. in 1995 based on the structural risk minimization (SRM) principle [34]. SVR (support vector regression), with good generalization and nonlinear processing ability, has been widely employed in hydrology [35–38]. The methodology of SVR is briefly described below.The basic idea of SVR for nonlinear case is to map the original problem to a linear problem in a high-dimensional feature space by nonlinear transformation to approximatef with input vector x∈Rn, output vector y∈R and sample set x1,y1,…,xl,yl:(6)fx=w,ϕx+b.In formula (6), f(x) is the regression function, ϕx is the nonlinear transformation function, w is the weight vector, and b is the threshold. Then, the ε-SVR model is built by solving the optimization problem:(7)minw,ξi,ξi∗,b12wTw+C1l∑i=1lξi+ξi∗,s.t.w,ϕxi+b−yi≤ε+ξi,yi−w,ϕxi+b≤ε+ξi∗,ξi,ξi∗≥0.,i=1,2,…,l.The dual form of the problem can be expressed as(8)maxα,α∗∑i=1lαiyi−ε−αi∗yi+ε−12∑i=1l∑j=1lαi−αi∗αj−αj∗Kxi⋅xj,s.t.∑i=1lαi−αi∗=0,0≤αi,αi∗≤C,i=1,…,l,where, Kxi,xj=ϕxi,ϕxj represents the kernel function. After the Lagrange multipliers, αi and αi∗, in formula (8) have been determined, the ε-SVR regression function f(x) can be established as(9)fx=∑SVsα¯i−α¯i∗Kxi,x+b.In this study, the radial basis function kernel with a parameterσ was selected for nonlinear transformation:(10)Kxi,x=exp−x−xi22σ2.The parameterε in formula (8) should be determined beforehand, but in many practical cases, it is hard to determine the value of ε before training. In order to solve this problem, a new parameter ν was introduced by Schölkopf (ν-SVR) [39]. Subsequently, the optimization problem can be changed as(11)minw,b,ξ,ξ∗ε12wTw+Cνε+1l∑i=1lξi+ξi∗,s.t.w,ϕxi+b−yi≤ε+ξi,yi−w,ϕxi+b≤ε+ξi∗,ξi,ξi∗>0,i=1,…,l,ε>0.Also, the dual form of formula (11) is(12)maxα,α∗∑i=1lαiyi−ε−αi∗yi+ε−12∑i=1l∑j=1lαi−αi∗αj−αj∗Kxi⋅xj,s.t.∑i=1lαi−αi∗=0,∑i=1lαi+αi∗≤C⋅ν,0≤αi,αi∗≤Cl,i=1,…,l.The capability ofν-SVR depends on the parameter C, ν, and σ. The cost constant C is a compromise between the complexity and generalization of the model. It is used to adjust the ratio of confidence range and empirical risk in the sample space, which determines the penalty degree for the sample whose loss is greater than ε. The parameter ν represents the lower bound of support vector and the upper bound of gap error. Additionally, since ν is introduced to optimize ε which controls the width of insensitive band and affects the number of support vectors, ν also indirectly affects the number of support vectors and prediction accuracy, and the reduction of ν leads to the reduction of support vectors. Accordingly, the quadratic programming problem (formula (12)) can be solved by determining the three parameters and the MAE of the ensemble forecast can be calculated by the following formula:(13)MAE=1l∑i=1lfxi−yi.In formula (13), f(xi) and yi indicate the ith ensemble forecast and observed values, and l is the sample size.Usually, in order to get the best approximation ability, nonlinear optimization algorithm is used to optimize the parametersC, ν, and σ by minimizing the MAE. However, the number of support vectors can be very large if the ν-SVR model is solved by optimization algorithm. Therefore, the range of parameter ν should be analyzed with determined C and σ by perturbation method.In this paper, theν-SVR model was used to approximate f and the parameters of SVR were optimized by the PSO (particle swarm optimization) algorithm with the objective function of the minimization of the MAE [40, 41]. Furthermore, the number of support vectors was controlled to be less than l/2 by controlling the range of ν.All the SVR models used in this research are based on the open source software LIBSVM developed by Lin Chih-Jen. More information about the model is available athttps://www.csie.ntu.edu.tw/∼cjlin/.


## 2.2.1. Bias-Removed Ensemble Mean

The bias-removed ensemble mean is developed from the ensemble mean, and the calculation formula is as follows:(4)F=O¯+1m∑i=1mFi−F¯i,where O¯ represents the average value of observations during the training period, m represents the number of members used in the ensemble forecast, Fi represents the forecast value of member i, F¯i represents the average value of member i during the training period, and F is the ensemble forecast value.


## 2.2.2. Support Vector Regression

In formula (4), the relationship between the ensemble forecast value F and the forecast value of member F1, F2,…, Fm is linear. The relationship can be considered to be nonlinear as follows:(5)F=fF1,F2,…,Fm.In formula (5), f represents the nonlinear relationship. A BP artificial neural network model or a support vector regression model can be used to approximate f.Support vector machine (SVM) is a mechanical learning method for classification and regression proposed by Vanpik et al. in 1995 based on the structural risk minimization (SRM) principle [34]. SVR (support vector regression), with good generalization and nonlinear processing ability, has been widely employed in hydrology [35–38]. The methodology of SVR is briefly described below.The basic idea of SVR for nonlinear case is to map the original problem to a linear problem in a high-dimensional feature space by nonlinear transformation to approximatef with input vector x∈Rn, output vector y∈R and sample set x1,y1,…,xl,yl:(6)fx=w,ϕx+b.In formula (6), f(x) is the regression function, ϕx is the nonlinear transformation function, w is the weight vector, and b is the threshold. Then, the ε-SVR model is built by solving the optimization problem:(7)minw,ξi,ξi∗,b12wTw+C1l∑i=1lξi+ξi∗,s.t.w,ϕxi+b−yi≤ε+ξi,yi−w,ϕxi+b≤ε+ξi∗,ξi,ξi∗≥0.,i=1,2,…,l.The dual form of the problem can be expressed as(8)maxα,α∗∑i=1lαiyi−ε−αi∗yi+ε−12∑i=1l∑j=1lαi−αi∗αj−αj∗Kxi⋅xj,s.t.∑i=1lαi−αi∗=0,0≤αi,αi∗≤C,i=1,…,l,where, Kxi,xj=ϕxi,ϕxj represents the kernel function. After the Lagrange multipliers, αi and αi∗, in formula (8) have been determined, the ε-SVR regression function f(x) can be established as(9)fx=∑SVsα¯i−α¯i∗Kxi,x+b.In this study, the radial basis function kernel with a parameterσ was selected for nonlinear transformation:(10)Kxi,x=exp−x−xi22σ2.The parameterε in formula (8) should be determined beforehand, but in many practical cases, it is hard to determine the value of ε before training. In order to solve this problem, a new parameter ν was introduced by Schölkopf (ν-SVR) [39]. Subsequently, the optimization problem can be changed as(11)minw,b,ξ,ξ∗ε12wTw+Cνε+1l∑i=1lξi+ξi∗,s.t.w,ϕxi+b−yi≤ε+ξi,yi−w,ϕxi+b≤ε+ξi∗,ξi,ξi∗>0,i=1,…,l,ε>0.Also, the dual form of formula (11) is(12)maxα,α∗∑i=1lαiyi−ε−αi∗yi+ε−12∑i=1l∑j=1lαi−αi∗αj−αj∗Kxi⋅xj,s.t.∑i=1lαi−αi∗=0,∑i=1lαi+αi∗≤C⋅ν,0≤αi,αi∗≤Cl,i=1,…,l.The capability ofν-SVR depends on the parameter C, ν, and σ. The cost constant C is a compromise between the complexity and generalization of the model. It is used to adjust the ratio of confidence range and empirical risk in the sample space, which determines the penalty degree for the sample whose loss is greater than ε. The parameter ν represents the lower bound of support vector and the upper bound of gap error. Additionally, since ν is introduced to optimize ε which controls the width of insensitive band and affects the number of support vectors, ν also indirectly affects the number of support vectors and prediction accuracy, and the reduction of ν leads to the reduction of support vectors. Accordingly, the quadratic programming problem (formula (12)) can be solved by determining the three parameters and the MAE of the ensemble forecast can be calculated by the following formula:(13)MAE=1l∑i=1lfxi−yi.In formula (13), f(xi) and yi indicate the ith ensemble forecast and observed values, and l is the sample size.Usually, in order to get the best approximation ability, nonlinear optimization algorithm is used to optimize the parametersC, ν, and σ by minimizing the MAE. However, the number of support vectors can be very large if the ν-SVR model is solved by optimization algorithm. Therefore, the range of parameter ν should be analyzed with determined C and σ by perturbation method.In this paper, theν-SVR model was used to approximate f and the parameters of SVR were optimized by the PSO (particle swarm optimization) algorithm with the objective function of the minimization of the MAE [40, 41]. Furthermore, the number of support vectors was controlled to be less than l/2 by controlling the range of ν.All the SVR models used in this research are based on the open source software LIBSVM developed by Lin Chih-Jen. More information about the model is available athttps://www.csie.ntu.edu.tw/∼cjlin/.


## 3. Data and Study Area

### 3.1. Study Area

The Huaihe River Basin is located in the middle east of China between the Yangtze River Basin and the Yellow River Basin, with an area of 270 thousand km2. Similar to many Chinese basins affected by the monsoon climate, the annual rainfall in the basin is uneven, with drought in spring and winter and rain in summer and autumn. The annual average rainfall of the basin is 875 mm, and the rainy season occurs from May to September, during which 50%–75% of the annual precipitation is concentrated. Therefore, floods are frequent and flood resources are abundant in the rainy season of the basin. However, only a small amount of flood resources is being used to solve the problem of water supply in the drought season. The continuous increase of freshwater demand in the basin has imposed new challenges on the flood control and hazard alleviation. The Xixian Catchment is one of the most important subcatchments in the upper Huaihe River Basin, which is located in the headstream area of the main stream of the Huaihe River. In this paper, the Xixian Catchment was selected as a typical subwatershed of the Huaihe River Basin to evaluate and correct precipitation forecasts of TIGGE, making it possible to improve the lead time of flood forecasts. Additionally, to meet the demand of flood forecasting, we divided the Xixian Catchment into six subcatchments, as shown in Figure 1.Figure 1
The Xixian Catchment in the upper Huaihe River Basin and the location of the six subcatchments.


### 3.2. Forecast Data and Observation Data

Eleven main operational forecast centers from different countries and regions participate in the TIGGE program, including the Bureau of Meteorology of Australia (BoM), China Meteorological Centre (CMA), Centre for Weather Forecasting and Climate Studies (CPTEC), Environment and Climate Change Canada (ECCC), European Centre for Medium-Range Weather Forecast (ECMWF), Japan Meteorological Agency (JMA), Korea Meteorological Agency (KMA), National Meteorological Service of France (Météo-France), United Kingdom Meteorological Office (UKMO), National Centers for Environmental Prediction (NCEP), and National Centre for Medium Range Weather Forecasting (NCMRWF). Five centers were selected in this study: CMA, JMA, KMA, ECMWF, and UKMO. The other six centers were not included in this study for various reasons. The data from BoM ended in 2010. As the location of CPTEC is in the Southern Hemisphere, its initial perturbations are not included in the Northern Hemisphere midlatitude climate. The dataset of the ECCC is missing several months in 2017. Météo-France only provides ensemble forecasts at 6 : 00 and 18 : 00 UTC. For NCEP, the dataset is missing many days in 2017. NCMRWF started providing data to TIGGE on August 2017. More details of the datasets used in this paper are briefly given in Table2.Table 2
Configurations of the five TIGGE ensemble forecasts investigated in this study.
Centre
Number of ensemble members
Horizontal resolution
Forecast length (h)
Base time (UTC)
Steps (h)
CMA
20 + 1
TL639
240
00:00; 12:00
6
JMA
50 + 1
T479
240
00:00; 12:00
6
KMA
23 + 1
N320
288
00:00; 12:00
6
ECMWF
50 + 1
TL639 (0–240 h) TL319 (240–360 h)
360
00:00; 12:00
6
UKMO
23 + 1
N640
360
00:00; 12:00
6As shown in Table2, on account of the differences in the ensemble forecasts of the 5 centers in terms of the horizontal resolution and forecast length, a series of methods were applied to make the datasets consistent. (1) We only selected the forecast with a base time at 00:00UTC; (2) the forecast length used in this paper was 168 h (7 days); and (3) the spatial resolutions of the centers were converted into 0.50 × 0.50. Additionally, several stations located in different subcatchments of the Xixian Catchment in the Huaihe River Basin were chosen as the validation dataset, as shown in Figure 1.


## 3.1. Study Area

The Huaihe River Basin is located in the middle east of China between the Yangtze River Basin and the Yellow River Basin, with an area of 270 thousand km2. Similar to many Chinese basins affected by the monsoon climate, the annual rainfall in the basin is uneven, with drought in spring and winter and rain in summer and autumn. The annual average rainfall of the basin is 875 mm, and the rainy season occurs from May to September, during which 50%–75% of the annual precipitation is concentrated. Therefore, floods are frequent and flood resources are abundant in the rainy season of the basin. However, only a small amount of flood resources is being used to solve the problem of water supply in the drought season. The continuous increase of freshwater demand in the basin has imposed new challenges on the flood control and hazard alleviation. The Xixian Catchment is one of the most important subcatchments in the upper Huaihe River Basin, which is located in the headstream area of the main stream of the Huaihe River. In this paper, the Xixian Catchment was selected as a typical subwatershed of the Huaihe River Basin to evaluate and correct precipitation forecasts of TIGGE, making it possible to improve the lead time of flood forecasts. Additionally, to meet the demand of flood forecasting, we divided the Xixian Catchment into six subcatchments, as shown in Figure 1.Figure 1
The Xixian Catchment in the upper Huaihe River Basin and the location of the six subcatchments.


## 3.2. Forecast Data and Observation Data

Eleven main operational forecast centers from different countries and regions participate in the TIGGE program, including the Bureau of Meteorology of Australia (BoM), China Meteorological Centre (CMA), Centre for Weather Forecasting and Climate Studies (CPTEC), Environment and Climate Change Canada (ECCC), European Centre for Medium-Range Weather Forecast (ECMWF), Japan Meteorological Agency (JMA), Korea Meteorological Agency (KMA), National Meteorological Service of France (Météo-France), United Kingdom Meteorological Office (UKMO), National Centers for Environmental Prediction (NCEP), and National Centre for Medium Range Weather Forecasting (NCMRWF). Five centers were selected in this study: CMA, JMA, KMA, ECMWF, and UKMO. The other six centers were not included in this study for various reasons. The data from BoM ended in 2010. As the location of CPTEC is in the Southern Hemisphere, its initial perturbations are not included in the Northern Hemisphere midlatitude climate. The dataset of the ECCC is missing several months in 2017. Météo-France only provides ensemble forecasts at 6 : 00 and 18 : 00 UTC. For NCEP, the dataset is missing many days in 2017. NCMRWF started providing data to TIGGE on August 2017. More details of the datasets used in this paper are briefly given in Table2.Table 2
Configurations of the five TIGGE ensemble forecasts investigated in this study.
Centre
Number of ensemble members
Horizontal resolution
Forecast length (h)
Base time (UTC)
Steps (h)
CMA
20 + 1
TL639
240
00:00; 12:00
6
JMA
50 + 1
T479
240
00:00; 12:00
6
KMA
23 + 1
N320
288
00:00; 12:00
6
ECMWF
50 + 1
TL639 (0–240 h) TL319 (240–360 h)
360
00:00; 12:00
6
UKMO
23 + 1
N640
360
00:00; 12:00
6As shown in Table2, on account of the differences in the ensemble forecasts of the 5 centers in terms of the horizontal resolution and forecast length, a series of methods were applied to make the datasets consistent. (1) We only selected the forecast with a base time at 00:00UTC; (2) the forecast length used in this paper was 168 h (7 days); and (3) the spatial resolutions of the centers were converted into 0.50 × 0.50. Additionally, several stations located in different subcatchments of the Xixian Catchment in the Huaihe River Basin were chosen as the validation dataset, as shown in Figure 1.


## 4. Results and Discussion

The results obtained are discussed in the four subsections below: (1) the evaluation of raw precipitation forecasting for different lead times and subcatchments; (2) the rain forecast skills of different ensemble methods; (3) a new ensemble method based on SVR and ME minimization; and (4) comparison of the performances obtained using different ensemble methods.


### 4.1. Evaluation of the Raw Forecasts

As a first step of this study, we estimated the accuracy of raw TIGGE precipitation forecasts from the selected five centers through their relationships with the measured data. The performance of the precipitation forecasts from different centers is illustrated in Figure2. Significantly, JMA has the best performances in all the subcatchments, especially for a longer lead time. For most of the subcatchments, the forecast skills of ECMWF are close to that of JMA when the lead time is less than 5 d, but the RMSE of ECMWF rises dramatically as the lead time becomes longer. In general, the prediction skills of ECMWF, UKMO, and JMA exhibit more obvious downward trends with the increase of the lead time, while the other two forecasts fluctuate at different lead times. Moreover, the performance of KMA in the study area is very unusual, as the RMSE of the +1 d forecast is almost two times that of other centers with the same lead time. However, the RMSE of KMA reduces to a normal level when the lead time increases, and it has the most volatile performance of the five datasets.Figure 2
The RMSEs of the forecasts in the six different subcatchments during May to September 2015–2017, where (a–f) represent subcatchments 1–6.
(a)
(b)
(c)
(d)
(e)
(f)The daily predicted rainfall versus observed rainfall during 2015–2017 in the whole Xixian Catchment for +1 d, +2 d, +5 d and +7 d is illustrated in Figure3. The low skill of KMA in the +1 d forecast is clearly identified in Figure 3(a), which is caused by the overestimates of rainfall. Table 3 lists the three rates of the Xixian Catchment for different lead times. From the table, the HRs vary inversely to the lead times, consistent with the results of Figure 2. In addition, it is obvious that all FARs with different lead times are much larger than the MARs, indicating that these forecasts may overestimate the actual rainfall overall. However, Figure 4 implies a converse conclusion for JMA and ECMWF in that the forecasts may underestimate the rainfall, as the outlier values of MA are more numerous and greater in value than those of FA. The results of forecast error distributions show that JMA and ECMWF underestimate the rainfall, while CMA, KMA, and UKMO overestimate the rainfall. Although the three rates and RMSEs indicate that JMA and ECMWF are the most skillful datasets in the Xixian Catchment, as mentioned above, they are still unfavorable for extending the lead time of flood forecasting. The analysis results prove that the raw rainfall forecast data cannot meet the demand for flood control, so it is necessary to improve the forecast ability by ensemble methods.Figure 3
Xixian catchment observations versus raw (a) +1 d (b) +2 d (c) +5 d and (d) +7 d forecasts for five centers during the rainy seasons in 2015 to 2017.
(a)
(b)
(c)
(d)Table 3
The three rates of the forecasts in Xixian catchment.
Lead time (days)
CMA
KMA
ECMWF
JMA
UKMO
HR
FAR
MAR
HR
FAR
MAR
HR
FAR
MAR
HR
FAR
MAR
HR
FAR
MAR
1
61.44
29.19
9.37
57.73
33.33
8.93
67.97
22.88
9.15
70.37
20.04
9.59
65.58
25.71
8.71
2
65.36
24.84
9.80
64.49
21.79
13.73
70.81
19.39
9.80
66.45
22.66
10.89
66.88
20.92
12.20
3
62.53
24.84
12.64
61.44
24.18
14.38
66.23
23.31
10.46
61.44
28.76
9.80
64.27
23.09
12.64
4
61.00
25.71
13.29
61.87
22.22
15.90
64.71
22.88
12.42
56.86
31.15
11.98
63.18
23.31
13.51
5
58.61
25.71
15.69
52.29
30.50
17.21
60.57
26.80
12.64
57.73
28.10
14.16
59.26
24.62
16.12
6
54.47
27.67
17.86
57.30
24.40
18.30
60.35
25.27
14.38
55.99
28.54
15.47
57.30
25.05
17.65
7
54.03
27.23
18.74
57.52
22.22
20.26
57.73
27.45
14.81
56.86
25.93
17.21
59.91
22.00
18.08Figure 4
Box and whisker plots of forecast errors in the Xixian watershed during 2015–2017. The cross represents the mean value, the horizontal line in the box represents the median of the distribution (50% of the data are greater than this value), and the upper and lower box limits represent the upper and lower quartiles (25% of data greater/lower than the value), respectively. Maximum and minimum values are indicated by the top and bottom horizontal lines. The outlier points show values more than two-thirds of the quantile. (a) +1 d. (b) +2 d. (c) +5 d. (d) +7 d
(a)
(b)
(c)
(d)


### 4.2. Evaluation of Ensemble Forecasts

As shown formerly, ensemble forecasting is a feasible way to reduce the errors of the initial field and system by assembling different datasets. In this paper, we employed the control forecast datasets from five centers for ensemble forecasting. Additionally, the performances of the ensemble methods were evaluated by the RMSE, three rates, and ME distributions from the aspects of the training and verification periods. Four different ensemble methods were used in this study, including both linear and nonlinear methods: EM, BREM, LR and SVR. The data during May to September 2015–2016 were selected as the training period, while the data in the flood season of 2017 were used for validation.The datasets of JMA were chosen as the best raw material, while those of CMA were selected as the worst for comparison with the ensemble forecasts during the training period. The four ensemble methods have better performances than CMA, and only few cases are inferior to JMA (Figure5), which proves that the ensemble prediction results are better than the original datasets. The RMSEs of EM and BREM are very close, and the differences in RMSEs between LR and the two are not large. In summary, the improvements of the three linear methods are extremely similar, that is, LR is slightly better than EM and BREM. SVR, as a nonlinear method, has a significantly better performance than linear methods in the training period. Additionally, since the accuracy of ensemble forecasts depends on the accuracy of the raw datasets, there is still an inverse relationship between the prediction accuracy and lead time.Figure 5
The RMSEs of different ensemble methods in the six different subcatchments during the training period, where (a–f) represent subcatchments 1–6.
(a)
(b)
(c)
(d)
(e)
(f)Table4 reveals a quite different conclusion that although the three linear methods perform similarly in RMSE, the HR of BREM is significantly higher than those of the other two methods, which have even lower HRs than the raw JMA dataset. Meanwhile, the HR of LR has a sharp decrease as the lead time reaches +6 d. This phenomenon is caused by the sudden drop of the hit number for no rain, from 127 at +5 d to 8 at +6 d. The main reason for the drop is that the linear equation enlarges all the values to the same extent due to the overall underestimation of the forecasts. Additionally, the SVR has the best result, with an increase of almost 15% in HR; however, compared with FAR, the decline of MAR is evidently smaller.Table 4
The three rates of the forecasts obtained using different ensemble methods in the Xixian catchment during the training period.
Lead time (days)
EM
BREM
LR
SVR
HR
FAR
MAR
HR
FAR
MAR
HR
FAR
MAR
HR
FAR
MAR
1
60.78
34.97
4.25
70.59
17.97
11.44
67.97
24.51
7.52
91.18
4.58
4.25
2
66.01
27.78
6.21
72.22
18.95
8.82
68.95
22.88
8.17
87.58
7.84
4.58
3
62.42
31.70
5.88
69.93
21.24
8.82
66.67
24.18
9.15
79.41
13.40
7.19
4
54.58
35.95
9.48
66.01
23.20
10.78
60.13
28.76
11.11
90.52
3.92
5.56
5
54.58
34.97
10.46
62.75
25.16
12.09
51.96
37.91
10.13
88.89
5.23
5.88
6
51.96
37.58
10.46
62.42
26.14
11.44
16.67
74.84
8.50
84.64
6.86
8.50
7
50.00
38.24
11.76
62.09
25.82
12.09
14.38
76.80
8.82
84.64
7.19
8.17Since this paper focuses on the application of numerical precipitation forecasts to improve the lead time of flood forecasts and flood control safety, the error of MAs (ME) in the forecasts should be taken into consideration. The ME distribution of the four methods in the Xixian Catchment for different lead times in the training period is presented in Figure6. The distribution of the four methods indicates the performance of the ensemble results in terms of correcting the ME. Both BREM and SVR have a range of MEs within 5 mm at different lead times during the training period, but there are still some large MEs, with a maximum of more than 70 mm. The performance of SVR at +3 d is acceptable for use in flood forecasting, with few MEs over 20 mm.Figure 6
Box and whisker plots showing the performance in terms of the ME for 1–7 d rainfall obtained using different methods in the Xixian Catchment during the training period. (a) FM. (b) PREM. (c) LR. (d) SVR.
(a)
(b)
(c)
(d)SVR has a remarkable advantage in promoting the accuracy of forecasts in the training period; however, many former cases suggested that some correction approaches perform well in the training period but have a poor performance in the validation period. Therefore, we evaluated the accuracy in the verification period in the Xixian Catchment from three aspects: the RMSE, MAR, and ME distributions.Additionally, the RMSE and three rates show different results in assessing the forecast ability during the verification period (Figure7 and Table 5). Figure 7 indicates that the performance of LR is worse than those of EM and BREM in the verification period, which has advantages in the training period. Moreover, the RMSE of SVR presents a sharp increase at +6 d, indicating the worst forecasting ability of the four methods, while the three rates show that this method has advantages at each lead time. Both views imply that SVR has the best performance within +5 d. In addition, we compared and analyzed the ME of each method to carry out a more comprehensive assessment.Figure 7
The RMSEs of different ensemble methods in the Xixian catchment during the verification period.Table 5
The three rates of the forecasts obtained using different ensemble methods in the Xixian catchment during the verification period.
Lead time (days)
EM
BREM
LR
SVR
HR
FAR
MAR
HR
FAR
MAR
HR
FAR
MAR
HR
FAR
MAR
1
67.97
22.88
9.15
78.43
7.19
14.38
68.63
18.95
12.42
81.70
10.46
7.84
2
67.97
24.18
7.84
74.51
16.99
8.50
68.63
20.26
11.11
73.20
14.38
12.42
3
52.94
37.91
9.15
66.01
24.18
9.80
58.82
27.45
13.73
71.90
18.30
9.80
4
56.86
33.33
9.80
66.67
21.57
11.76
59.48
26.80
13.73
77.12
11.11
11.76
5
54.25
34.64
11.11
60.78
26.14
13.07
52.94
32.03
15.03
73.86
15.69
10.46
6
50.98
34.64
14.38
63.40
21.57
15.03
7.84
74.51
17.65
75.82
12.42
11.76
7
47.06
36.60
16.34
57.52
24.84
17.65
15.69
69.28
15.03
71.90
12.42
15.69The ME distribution of BREM is the most uniform during the verification period (Figure8), with most of the MEs between 0 and 5 mm except for the outlier values. However, the outlier values of BREM are greater than those of SVR, which is disadvantageous to flood control safety. Although the ME distribution ranges of SVR are larger, its outlier values are fewer in number and smaller in value within +3 d. To sum up, for the three linear methods, BREM has an obvious advantage over the other two, but the nonlinear method SVR is better than BREM. The result of SVR presents high precision in terms of the RMSE and three rates, but the distribution of the ME indicates that it still cannot avoid the huge disadvantages conferred to flood control security caused by MAs. Consequently, it is necessary to find a new ensemble method with smaller ME for flood forecast.Figure 8
Box and whisker plots showing the performance in terms of the ME for 1–7 d rainfall forecasts obtained using different methods in the Xixian catchment during the verification period. (a) FM. (b) PREM. (c) LR. (d) SVR.
(a)
(b)
(c)
(d)


### 4.3. A New Ensemble Method for MA Rate Minimization

Although the results of SVR ensemble forecasting have suggested that it can effectively improve the accuracy of precipitation forecasts from the aspect of RMSE and the three rates, there are still large MEs (around 50 mm), which may generate great risk in flood control. However, since the FE may only lead to loss of economic benefits, while the ME may cause MAs of floods along with gigantic losses of life and economic damage, the results still cannot meet the requirement of prolonging the lead time of flood forecast without bringing huge flood risk. The traditional ensemble methods with minimizing MAE considers the whole deviation, which implies equal weights of ME and FE, but it is not appropriate for flood control where safety is the most important. Therefore, the ME should be reduced as much as possible in the ensemble forecast, and a modest loss of FE is acceptable. For this purpose, a new objection function was proposed for theν-SVR model by minimizing the RMSE of MAs (SVR-MA).(14)RMSEma=1m∑i=1lDi2,(15)Di=yi−fxi,fxi<yi,0,fxi≥yi,(16)m=∑i=1lIDi,IDi=1,Di>0,0,Di≤0,wheref(xi) represents the function established by ν-SVR. Here, formula (14) was employed to optimize the parameters C, σ, and ν by minimizing the ME instead of the MAE in formula (13). Also, the number of support vector was controlled around 2/l by perturbing ν after the optimization.Additionally, the improvement of the new method was analyzed compared with SVR and BREM for both the training period and the verification period. The RMSEs of SVR and SVR-MA are obviously smaller than that of BREM, and the RMSE of SVR-MA implies the best performance of the three methods within +3 d (Figure9). From the aspect of the three rates (Table 6), all the HRs from different lead times are less than 50%, with a sharp increase in FAR, while the decrease of MAR is not clear. Table 7 illustrates the numbers of hits, MAs, and FAs from +1 d in the Xixian Watershed. Compared with the original SVR, the number of hits at the no rain magnitude using SVR-MA is far less than that with SVR, while the FAR number has a dramatic increase at light rain. Additionally, the MA number presents a slight decrease at each magnitude. For the purpose of flood control safety, it is a viable solution to exchange the accuracy of some FAs for the decrement in the MA number, especially for the FAs at no rain, which have a limited influence on the benefits of water resource management.Figure 9
The RMSEs of different ensemble methods in the Xixian catchment. (a) Training period. (b) Verification period.
(a)
(b)Table 6
The three rates of SVR-MA in the Xixian catchment.
Lead time (days)
Training period
Verification period
HR
FAR
MAR
HR
FAR
MAR
1
47.71
49.02
3.27
39.22
52.29
8.50
2
78.10
16.99
4.90
71.90
20.92
7.19
3
32.68
64.05
3.27
26.14
64.05
9.80
4
49.67
44.12
6.21
43.14
47.71
9.15
5
36.60
59.48
3.92
26.14
61.44
12.42
6
33.01
61.76
5.23
26.80
62.09
11.11
7
33.99
60.78
5.23
27.98
63.62
8.50Table 7
The numbers of hit, FAs, and MAs in the Xixian catchment obtained by different methods at +1 d during the whole period.
Magnitude
SVR-RM
SVR
Hit
FA
MA
Hit
FA
MA
No rain
121
0
11
320
0
16
Light rain
43
220
5
43
23
5
Medium rain
23
6
4
20
4
3
Heavy rain
15
4
3
16
3
1
Rainstorm
4
0
0
5
0
0
Heavy rainstorm
0
0
0
0
0
0
Extreme rainstorm
0
0
0
0
0
0As we have discussed in the preceding part of this paper, it is biased to evaluate the rainfall forecast using only the three rates and RMSE. The distribution of MEs obtained using SVR-MA with different lead times is shown in Figure10. The performance of SVR-MA is better than that of SVR, with a smaller distribution range and fewer outlier values for both the training period and verification period, especially when the lead time is 2 d. As most of the distribution ranges of MEs within +3 d in the verification period are less than 15 mm, several outlier values are expected. Moreover, most of the outlier values within +3 d are less than 30 mm.Figure 10
Box and whisker plots showing the distribution of the ME for 1–7 d rainfall forecasts obtained by SVR-MA in the Xixian Catchment. (a) Training period. (b) Verification period.
(a)
(b)In conclusion, compared with the originalν-SVR, the new ensemble method (SVR-MA) has a similar performance in RMSE and three rates, which proves that the new model still has good approximation and generation ability. Meanwhile, SVR-MA is an efficient way to reduce the number and value of ME, especially for the number of ME over 20 mm. Most of the MEs are less than 10 mm in 3 d, and the maximum is around 30 mm. The new model not only reduces the overall error, but also makes the ME under special control, which is beneficial to flood control safety and can be applied to prolong the forecast period of flood forecasting with small flood control risk.


## 4.1. Evaluation of the Raw Forecasts

As a first step of this study, we estimated the accuracy of raw TIGGE precipitation forecasts from the selected five centers through their relationships with the measured data. The performance of the precipitation forecasts from different centers is illustrated in Figure2. Significantly, JMA has the best performances in all the subcatchments, especially for a longer lead time. For most of the subcatchments, the forecast skills of ECMWF are close to that of JMA when the lead time is less than 5 d, but the RMSE of ECMWF rises dramatically as the lead time becomes longer. In general, the prediction skills of ECMWF, UKMO, and JMA exhibit more obvious downward trends with the increase of the lead time, while the other two forecasts fluctuate at different lead times. Moreover, the performance of KMA in the study area is very unusual, as the RMSE of the +1 d forecast is almost two times that of other centers with the same lead time. However, the RMSE of KMA reduces to a normal level when the lead time increases, and it has the most volatile performance of the five datasets.Figure 2
The RMSEs of the forecasts in the six different subcatchments during May to September 2015–2017, where (a–f) represent subcatchments 1–6.
(a)
(b)
(c)
(d)
(e)
(f)The daily predicted rainfall versus observed rainfall during 2015–2017 in the whole Xixian Catchment for +1 d, +2 d, +5 d and +7 d is illustrated in Figure3. The low skill of KMA in the +1 d forecast is clearly identified in Figure 3(a), which is caused by the overestimates of rainfall. Table 3 lists the three rates of the Xixian Catchment for different lead times. From the table, the HRs vary inversely to the lead times, consistent with the results of Figure 2. In addition, it is obvious that all FARs with different lead times are much larger than the MARs, indicating that these forecasts may overestimate the actual rainfall overall. However, Figure 4 implies a converse conclusion for JMA and ECMWF in that the forecasts may underestimate the rainfall, as the outlier values of MA are more numerous and greater in value than those of FA. The results of forecast error distributions show that JMA and ECMWF underestimate the rainfall, while CMA, KMA, and UKMO overestimate the rainfall. Although the three rates and RMSEs indicate that JMA and ECMWF are the most skillful datasets in the Xixian Catchment, as mentioned above, they are still unfavorable for extending the lead time of flood forecasting. The analysis results prove that the raw rainfall forecast data cannot meet the demand for flood control, so it is necessary to improve the forecast ability by ensemble methods.Figure 3
Xixian catchment observations versus raw (a) +1 d (b) +2 d (c) +5 d and (d) +7 d forecasts for five centers during the rainy seasons in 2015 to 2017.
(a)
(b)
(c)
(d)Table 3
The three rates of the forecasts in Xixian catchment.
Lead time (days)
CMA
KMA
ECMWF
JMA
UKMO
HR
FAR
MAR
HR
FAR
MAR
HR
FAR
MAR
HR
FAR
MAR
HR
FAR
MAR
1
61.44
29.19
9.37
57.73
33.33
8.93
67.97
22.88
9.15
70.37
20.04
9.59
65.58
25.71
8.71
2
65.36
24.84
9.80
64.49
21.79
13.73
70.81
19.39
9.80
66.45
22.66
10.89
66.88
20.92
12.20
3
62.53
24.84
12.64
61.44
24.18
14.38
66.23
23.31
10.46
61.44
28.76
9.80
64.27
23.09
12.64
4
61.00
25.71
13.29
61.87
22.22
15.90
64.71
22.88
12.42
56.86
31.15
11.98
63.18
23.31
13.51
5
58.61
25.71
15.69
52.29
30.50
17.21
60.57
26.80
12.64
57.73
28.10
14.16
59.26
24.62
16.12
6
54.47
27.67
17.86
57.30
24.40
18.30
60.35
25.27
14.38
55.99
28.54
15.47
57.30
25.05
17.65
7
54.03
27.23
18.74
57.52
22.22
20.26
57.73
27.45
14.81
56.86
25.93
17.21
59.91
22.00
18.08Figure 4
Box and whisker plots of forecast errors in the Xixian watershed during 2015–2017. The cross represents the mean value, the horizontal line in the box represents the median of the distribution (50% of the data are greater than this value), and the upper and lower box limits represent the upper and lower quartiles (25% of data greater/lower than the value), respectively. Maximum and minimum values are indicated by the top and bottom horizontal lines. The outlier points show values more than two-thirds of the quantile. (a) +1 d. (b) +2 d. (c) +5 d. (d) +7 d
(a)
(b)
(c)
(d)


## 4.2. Evaluation of Ensemble Forecasts

As shown formerly, ensemble forecasting is a feasible way to reduce the errors of the initial field and system by assembling different datasets. In this paper, we employed the control forecast datasets from five centers for ensemble forecasting. Additionally, the performances of the ensemble methods were evaluated by the RMSE, three rates, and ME distributions from the aspects of the training and verification periods. Four different ensemble methods were used in this study, including both linear and nonlinear methods: EM, BREM, LR and SVR. The data during May to September 2015–2016 were selected as the training period, while the data in the flood season of 2017 were used for validation.The datasets of JMA were chosen as the best raw material, while those of CMA were selected as the worst for comparison with the ensemble forecasts during the training period. The four ensemble methods have better performances than CMA, and only few cases are inferior to JMA (Figure5), which proves that the ensemble prediction results are better than the original datasets. The RMSEs of EM and BREM are very close, and the differences in RMSEs between LR and the two are not large. In summary, the improvements of the three linear methods are extremely similar, that is, LR is slightly better than EM and BREM. SVR, as a nonlinear method, has a significantly better performance than linear methods in the training period. Additionally, since the accuracy of ensemble forecasts depends on the accuracy of the raw datasets, there is still an inverse relationship between the prediction accuracy and lead time.Figure 5
The RMSEs of different ensemble methods in the six different subcatchments during the training period, where (a–f) represent subcatchments 1–6.
(a)
(b)
(c)
(d)
(e)
(f)Table4 reveals a quite different conclusion that although the three linear methods perform similarly in RMSE, the HR of BREM is significantly higher than those of the other two methods, which have even lower HRs than the raw JMA dataset. Meanwhile, the HR of LR has a sharp decrease as the lead time reaches +6 d. This phenomenon is caused by the sudden drop of the hit number for no rain, from 127 at +5 d to 8 at +6 d. The main reason for the drop is that the linear equation enlarges all the values to the same extent due to the overall underestimation of the forecasts. Additionally, the SVR has the best result, with an increase of almost 15% in HR; however, compared with FAR, the decline of MAR is evidently smaller.Table 4
The three rates of the forecasts obtained using different ensemble methods in the Xixian catchment during the training period.
Lead time (days)
EM
BREM
LR
SVR
HR
FAR
MAR
HR
FAR
MAR
HR
FAR
MAR
HR
FAR
MAR
1
60.78
34.97
4.25
70.59
17.97
11.44
67.97
24.51
7.52
91.18
4.58
4.25
2
66.01
27.78
6.21
72.22
18.95
8.82
68.95
22.88
8.17
87.58
7.84
4.58
3
62.42
31.70
5.88
69.93
21.24
8.82
66.67
24.18
9.15
79.41
13.40
7.19
4
54.58
35.95
9.48
66.01
23.20
10.78
60.13
28.76
11.11
90.52
3.92
5.56
5
54.58
34.97
10.46
62.75
25.16
12.09
51.96
37.91
10.13
88.89
5.23
5.88
6
51.96
37.58
10.46
62.42
26.14
11.44
16.67
74.84
8.50
84.64
6.86
8.50
7
50.00
38.24
11.76
62.09
25.82
12.09
14.38
76.80
8.82
84.64
7.19
8.17Since this paper focuses on the application of numerical precipitation forecasts to improve the lead time of flood forecasts and flood control safety, the error of MAs (ME) in the forecasts should be taken into consideration. The ME distribution of the four methods in the Xixian Catchment for different lead times in the training period is presented in Figure6. The distribution of the four methods indicates the performance of the ensemble results in terms of correcting the ME. Both BREM and SVR have a range of MEs within 5 mm at different lead times during the training period, but there are still some large MEs, with a maximum of more than 70 mm. The performance of SVR at +3 d is acceptable for use in flood forecasting, with few MEs over 20 mm.Figure 6
Box and whisker plots showing the performance in terms of the ME for 1–7 d rainfall obtained using different methods in the Xixian Catchment during the training period. (a) FM. (b) PREM. (c) LR. (d) SVR.
(a)
(b)
(c)
(d)SVR has a remarkable advantage in promoting the accuracy of forecasts in the training period; however, many former cases suggested that some correction approaches perform well in the training period but have a poor performance in the validation period. Therefore, we evaluated the accuracy in the verification period in the Xixian Catchment from three aspects: the RMSE, MAR, and ME distributions.Additionally, the RMSE and three rates show different results in assessing the forecast ability during the verification period (Figure7 and Table 5). Figure 7 indicates that the performance of LR is worse than those of EM and BREM in the verification period, which has advantages in the training period. Moreover, the RMSE of SVR presents a sharp increase at +6 d, indicating the worst forecasting ability of the four methods, while the three rates show that this method has advantages at each lead time. Both views imply that SVR has the best performance within +5 d. In addition, we compared and analyzed the ME of each method to carry out a more comprehensive assessment.Figure 7
The RMSEs of different ensemble methods in the Xixian catchment during the verification period.Table 5
The three rates of the forecasts obtained using different ensemble methods in the Xixian catchment during the verification period.
Lead time (days)
EM
BREM
LR
SVR
HR
FAR
MAR
HR
FAR
MAR
HR
FAR
MAR
HR
FAR
MAR
1
67.97
22.88
9.15
78.43
7.19
14.38
68.63
18.95
12.42
81.70
10.46
7.84
2
67.97
24.18
7.84
74.51
16.99
8.50
68.63
20.26
11.11
73.20
14.38
12.42
3
52.94
37.91
9.15
66.01
24.18
9.80
58.82
27.45
13.73
71.90
18.30
9.80
4
56.86
33.33
9.80
66.67
21.57
11.76
59.48
26.80
13.73
77.12
11.11
11.76
5
54.25
34.64
11.11
60.78
26.14
13.07
52.94
32.03
15.03
73.86
15.69
10.46
6
50.98
34.64
14.38
63.40
21.57
15.03
7.84
74.51
17.65
75.82
12.42
11.76
7
47.06
36.60
16.34
57.52
24.84
17.65
15.69
69.28
15.03
71.90
12.42
15.69The ME distribution of BREM is the most uniform during the verification period (Figure8), with most of the MEs between 0 and 5 mm except for the outlier values. However, the outlier values of BREM are greater than those of SVR, which is disadvantageous to flood control safety. Although the ME distribution ranges of SVR are larger, its outlier values are fewer in number and smaller in value within +3 d. To sum up, for the three linear methods, BREM has an obvious advantage over the other two, but the nonlinear method SVR is better than BREM. The result of SVR presents high precision in terms of the RMSE and three rates, but the distribution of the ME indicates that it still cannot avoid the huge disadvantages conferred to flood control security caused by MAs. Consequently, it is necessary to find a new ensemble method with smaller ME for flood forecast.Figure 8
Box and whisker plots showing the performance in terms of the ME for 1–7 d rainfall forecasts obtained using different methods in the Xixian catchment during the verification period. (a) FM. (b) PREM. (c) LR. (d) SVR.
(a)
(b)
(c)
(d)


## 4.3. A New Ensemble Method for MA Rate Minimization

Although the results of SVR ensemble forecasting have suggested that it can effectively improve the accuracy of precipitation forecasts from the aspect of RMSE and the three rates, there are still large MEs (around 50 mm), which may generate great risk in flood control. However, since the FE may only lead to loss of economic benefits, while the ME may cause MAs of floods along with gigantic losses of life and economic damage, the results still cannot meet the requirement of prolonging the lead time of flood forecast without bringing huge flood risk. The traditional ensemble methods with minimizing MAE considers the whole deviation, which implies equal weights of ME and FE, but it is not appropriate for flood control where safety is the most important. Therefore, the ME should be reduced as much as possible in the ensemble forecast, and a modest loss of FE is acceptable. For this purpose, a new objection function was proposed for theν-SVR model by minimizing the RMSE of MAs (SVR-MA).(14)RMSEma=1m∑i=1lDi2,(15)Di=yi−fxi,fxi<yi,0,fxi≥yi,(16)m=∑i=1lIDi,IDi=1,Di>0,0,Di≤0,wheref(xi) represents the function established by ν-SVR. Here, formula (14) was employed to optimize the parameters C, σ, and ν by minimizing the ME instead of the MAE in formula (13). Also, the number of support vector was controlled around 2/l by perturbing ν after the optimization.Additionally, the improvement of the new method was analyzed compared with SVR and BREM for both the training period and the verification period. The RMSEs of SVR and SVR-MA are obviously smaller than that of BREM, and the RMSE of SVR-MA implies the best performance of the three methods within +3 d (Figure9). From the aspect of the three rates (Table 6), all the HRs from different lead times are less than 50%, with a sharp increase in FAR, while the decrease of MAR is not clear. Table 7 illustrates the numbers of hits, MAs, and FAs from +1 d in the Xixian Watershed. Compared with the original SVR, the number of hits at the no rain magnitude using SVR-MA is far less than that with SVR, while the FAR number has a dramatic increase at light rain. Additionally, the MA number presents a slight decrease at each magnitude. For the purpose of flood control safety, it is a viable solution to exchange the accuracy of some FAs for the decrement in the MA number, especially for the FAs at no rain, which have a limited influence on the benefits of water resource management.Figure 9
The RMSEs of different ensemble methods in the Xixian catchment. (a) Training period. (b) Verification period.
(a)
(b)Table 6
The three rates of SVR-MA in the Xixian catchment.
Lead time (days)
Training period
Verification period
HR
FAR
MAR
HR
FAR
MAR
1
47.71
49.02
3.27
39.22
52.29
8.50
2
78.10
16.99
4.90
71.90
20.92
7.19
3
32.68
64.05
3.27
26.14
64.05
9.80
4
49.67
44.12
6.21
43.14
47.71
9.15
5
36.60
59.48
3.92
26.14
61.44
12.42
6
33.01
61.76
5.23
26.80
62.09
11.11
7
33.99
60.78
5.23
27.98
63.62
8.50Table 7
The numbers of hit, FAs, and MAs in the Xixian catchment obtained by different methods at +1 d during the whole period.
Magnitude
SVR-RM
SVR
Hit
FA
MA
Hit
FA
MA
No rain
121
0
11
320
0
16
Light rain
43
220
5
43
23
5
Medium rain
23
6
4
20
4
3
Heavy rain
15
4
3
16
3
1
Rainstorm
4
0
0
5
0
0
Heavy rainstorm
0
0
0
0
0
0
Extreme rainstorm
0
0
0
0
0
0As we have discussed in the preceding part of this paper, it is biased to evaluate the rainfall forecast using only the three rates and RMSE. The distribution of MEs obtained using SVR-MA with different lead times is shown in Figure10. The performance of SVR-MA is better than that of SVR, with a smaller distribution range and fewer outlier values for both the training period and verification period, especially when the lead time is 2 d. As most of the distribution ranges of MEs within +3 d in the verification period are less than 15 mm, several outlier values are expected. Moreover, most of the outlier values within +3 d are less than 30 mm.Figure 10
Box and whisker plots showing the distribution of the ME for 1–7 d rainfall forecasts obtained by SVR-MA in the Xixian Catchment. (a) Training period. (b) Verification period.
(a)
(b)In conclusion, compared with the originalν-SVR, the new ensemble method (SVR-MA) has a similar performance in RMSE and three rates, which proves that the new model still has good approximation and generation ability. Meanwhile, SVR-MA is an efficient way to reduce the number and value of ME, especially for the number of ME over 20 mm. Most of the MEs are less than 10 mm in 3 d, and the maximum is around 30 mm. The new model not only reduces the overall error, but also makes the ME under special control, which is beneficial to flood control safety and can be applied to prolong the forecast period of flood forecasting with small flood control risk.


## 5. Conclusions

How to extend the lead time of flood forecasts is essential for flood control safety and the development of society and the economy. Although numerical forecasting provides a possible way to solve this issue, it is still an urgent problem to fairly evaluate the accuracy of a rainfall forecast. Simple summary scores and classifications have limitations and cannot reflect the risk to flood control that precipitation forecasts may produce. In addition, more attention should be paid to the different impacts of MAs and FAs for flood control safety. In this study, we evaluated the ability of meteorological models of TIGGE to forecast the rainfall in six subcatchments with small areas during the flood seasons. Five numerical datasets were retained over the rainy seasons of 2015–2017 and compared with the observed dataset. We focused on the possibility for using rainfall forecast datasets to improve the lead time of flood forecasting. Additionally, we attempted to improve the performance of precipitation forecasts by employing different multimodal ensemble methods.The forecast skill of most datasets from TIGGE has an inverse relationship with the lead time, which is in accordance with previous studies. Comparison between different datasets is undertaken from different aspects to evaluate the accuracy of the rainfall forecasts. The results show that the forecasts of JMA have the best performances in each subcatchment, and KMA and CMA exhibit irregular fluctuations at different lead times. Although the HRs of rainfall forecasts from the five centers are over 50% at +7 d, with high overestimations in CMA, KMA, and UKMO, as well as obvious underestimations by JMA and ECMWF, the raw datasets cannot meet the demand for flood forecasting.Four ensemble methods were used in this study, including both linear and nonlinear methods. The results of SVR within 5 d are more reliable and accurate in comparison to those of the other three linear methods. Since, in this study, we placed more emphasis on the decrement of the ME, which leads to the risk of flood control, a new ensemble approach was proposed based on SVR. Compared with the original SVR, SVR-MA has a better performance in terms of ME at the expense of the FE, which is linked to the loss of benefits. However, most of the newly increased FAs are augmented from the no rain to light rain levels, which have a relatively low influence on the efficiency of water resource management. Meanwhile, we found that the performance of SVR-MA depends more on the accuracy of the datasets; thus, the results of SVR-MA after +3 d exhibit low skill in terms of the ME. It is worth mentioning that SVR-MA helps correcting the rainfall forecast and the ME of SVR-MA within +3 d may be an acceptable result for flood forecasting without causing much extra risk. These results provide us a new way to improve the lead time of flood forecasts under the prerequisite of flood control safety.



---

*Source: 7809302-2018-10-23.xml*