# Evaluation and Correction of Quantitative Precipitation Forecast by Storm-Scale NWP Model in Jiangsu, China

Authors: Gaili Wang; Dan Wang; Ji Yang; Liping Liu
Journal: Advances in Meteorology
 (2016)

---

## Abstract

With the development of high-performance computer systems and data assimilation techniques, storm-scale numerical weather prediction (NWP) models are gradually used for short-term deterministic forecasts. The primary objective of this study is to evaluate and correct precipitation forecasts of a storm-scale NWP model called the advanced regional prediction system (ARPS). The evaluation and correction consider five heavy precipitation events that occurred in the summer of 2015 in Jiangsu, China. The performances of the original and corrected ARPS precipitation forecasts are evaluated as a function of lead time using standard measurements and a spatial verification method called Structure-Amplitude-Location (SAL). In general, the ARPS could not produce optimal forecasts for very short lead times, and the forecast accuracy improves with increasing lead time. The ARPS overestimates precipitation for all lead times, which is confirmed by large bias in many forecasts in the first and second quadrant of the diagram of SAL, especially at the 1 h lead time. The amplitude correction is performed by matching percentile values of the ARPS precipitation forecasts and observations for each lead time. Amplitude correction significantly improved the ARPS precipitation forecasts in terms of the considered performance indices of standard measures and A-component and S-component of SAL.


---

## Body

## 1. Introduction

Heavy rain is one of the most severe weather events in China, causing floods and other geological and hydrological disasters. High-resolution quantitative precipitation forecasts (QPFs) play an important role in flash flood warning and emergency response.NWP models with atmospheric dynamic constraints have been used to operate for middle and long term weather forecast. However, the accuracy of NWP model forecasts during the first few hours is always influenced by the “spin-up” problem [1]. Therefore, precipitation forecasts of NWP models are less accurate than predictions of extrapolation-based techniques at short-term lead times [2]. Recently, with the development of high-performance computers and the use of rapid-update-cycle (RUC) approach, the “spin-up” problem of NWP models has been significantly reduced. The forecast accuracy at the first several hours has been improved significantly by assimilating various types of observation data [3–9]. NWP models with high spatial and temporal resolution have been applied for nowcasting of precipitation gradually. The high-resolution rapid refresh [10] developed by the National Oceanic and Atmospheric Administration, Weather Research and Forecasting model (WRF) with RUC technique used in the Beijing Meteorological Bureau, Chinese Meteorological Administration, and the advanced regional prediction system (ARPS) developed by the Center for Analysis and Prediction of Storms (CAPS) have been operationally applied for nowcasting precipitation.The Jiangsu Observatory introduced the ARPS model from CAPS to provide high quality and resolution weather forecast services during the second Youth Olympics Games held in Nanjing, China, in 2014. The purpose of this paper is to evaluate and correct the ARPS precipitation nowcasting in Jiangsu, China. The evaluation and correction are performed for hourly precipitation forecasts at lead times from 1 to 6 h. The eight convective heavy rain events that occurred during the summer of 2014 and 2015 in Jiangsu are selected in this study. Three heavy precipitation events occurring during the summer of 2014 were used for deriving the ARPS model correction parameters. Five heavy precipitation events occurring during the summers of 2014 and 2015 were used for verification data. The two verification events occurring in April 2014 were convective precipitation and lasted for about one day. The three verification events in 2015 lasted for several days and were accompanied by floods that cause significant economic losses and even casualties. The forecast of such events is beneficial for preventing disasters and reducing damage.The paper is organized as follows. Section2 describes data used in this study. The ARPS model and correction scheme are introduced in Sections 3 and 4, respectively. The evaluation methods are described in Section 5, and the results of the evaluation the ARPS original and corrected precipitation forecasts based on the five verification heavy rain events are presented in Section 6. The conclusions and discussions are drawn in Section 7.


## 2. Data and Case Study

In this study, quantitative precipitation estimations (QPEs) based radar is used to evaluate the original and corrected ARPS precipitation forecasts and derive the calibration parameters of the ARPS model. Figure1 gives the study domain, in which six single polarization S-band radar positions are marked with triangles. Radar data were measured in standard precipitation mode of 9 elevation scans with 6 min by the Chinese Meteorological Administration (CMA) radar network. Radar data are expressed in spherical coordinates (elevation, azimuth, and gate), with radial resolution of 1 km and azimuth resolution of 1°. The single radar data underwent quality control to reduce ground clutter, electronic interference, and anomalous propagations by a fuzzy logic algorithm [11]. The single radar data were merged using an exponential weighting average scheme to yield the mosaic reflectivity value in the overlapping coverage areas [12]. Rainfall rates are calculated using the local radar reflectivity-rainfall rate (Z-R) relationship of Z
=
386
R
1.43 at an elevation of 3 km above sea level with a spatial resolution of 0.01°. The rain rates are integrated over time to calculate hourly precipitation [13]. Due to asymmetric distribution of precipitation, the raindrop spectrum changes with time, space, and different types of precipitation. The fixed Z-R relationship results in the radar-based QPEs underestimation for light rain and basic satisfaction for heavy rain. Because heavy rain often occurs in summer in Jiangsu, in this study, the radar-based hourly QPEs were used as observations for the verification and correction of the ARPS forecasts.Figure 1
The study domain and locations (as triangles) of Doppler radars.Because Jiangsu is located in a transitional climate zone between subtropics and warm temperate zone, continuous heavy precipitation events often occur in June and July every year called Meiyu. The flood disaster in Jianghuai River caused by Meiyu is one of important meteorological disasters. That is why the forecast of such events is beneficial. Therefore, eight convective heavy rain events in the summer of 2014 and 2015 in Jiangsu are selected in this study. Figure2 shows one representative image of each event. Three heavy precipitation events occurred during the summer of 2014 as calibration data were used for deriving the ARPS model correction parameters. The other five heavy precipitation events were used to evaluate the original and corrected ARPS precipitation forecasts. Among the five heavy rain events, the disaster from 23rd of June 2015 heavy rain event was the most serious, affecting 93.5 million people, with one death, and economic losses of 2.1 billion RMB.Figure 2
Precipitation images of 1 h QPE of six heavy rain events used in this study. Image at 0400 BJT on 1st of June 2014 for event 1 (a), image at 2300 BJT on 16th of June 2014 for event 2 (b), image at 0100 BJT on 13th of August 2014 for event 3 (c), image at 0900 BJT on 16th of June 2015 for event 4 (d), image at 1500 BJT on 24th of June 2015 for event 5 (e), image at 1200 BJT on 6th of July 2015 for event 6 (f), image at 0900 BJT on 11th of April 2014 for event 7 (g), and image at 2100 BJT on 17th of April 2014 for event 8 (h).
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)


## 3. Advanced Regional Prediction System

The Advanced Regional Prediction System (ARPS) is developed at the Center for Analysis and Prediction of Storms at the University of Oklahoma and suitable to explicitly predict storm-scale convective systems as well as other scales weather systems. The ARPS is a nonhydrostatic compressible model and includes its own data ingest, quality control, and objective analysis packages, a three-dimensional variational (3D-Var) system, the forward prediction component, and a self-contained postprocessing, diagnostic and verification package [14–16]. The ARPS could predict 3D velocity vector (u, v, and w), pressure (p), turbulence kinetic energy (TKE), potential temperature (θ), water vapor mixing ratio (q
v), and the mixing ratios of cloud water, rainwater, ice, snow, and hail (q
c, q
r, q
i, q
s, and q
h, resp.). In the ARPS, subgrid-scale turbulent mixing is handled by three subgrid-scale closure schemes: first-order Smagorinsky/Lilly scheme, the 1.5-order TKE-based scheme, and the Germano dynamic closure scheme [17–20]. The combination of the 3D, 1.5-order TKE-based turbulence scheme and an ensemble turbulence closure scheme of Sun and Chang [21] is used to treat convective boundary layer turbulence.For the precipitation processes, the Kessler two-category liquid water scheme and the modified three-category ice schemes are used in the ARPS [22]. A fourth-order monotonic flux-corrected transport scheme [23] is applied to potential temperature, water variables, and TKE. Details on these physics and computational options can be found in Xue et al., [14–16].To obtain the initial conditions, six S-band radar observations in Jiangsu, including Doppler velocity and reflectivity factor, are assimilated with a 3D variational cloud analysis system in the ARPS [24–27]. The ARPS has been operating in Jiangsu Observatory and initialized every 3 hours since 2014. The ARPS produces forecasts up to 24 h ahead, with high spatial resolution of 3 km × 3 km and temporal interval of 1 h. Considering the importance and difficulty of precipitation nowcasting, the ARPS original and corrected forecast precipitation for the next 6 hours was evaluated and compared in this paper.


## 4. Correction of ARPS Forecasts

Hoffman and Grassotti [28] decomposed forecast error into displacement error, amplitude error, and residual error; moreover, displacement error and amplitude error must be large scale. The displacement error and amplitude error could be reduced by analyzing and correcting the differences between the forecast fields and observations [25, 29]. For the ARPS, considering the amplitude error is more remarkable than displacement error, an amplitude-correcting scheme was applied to improve forecast accuracy of the ARPS in this study.Precipitation is unarguably the vital input data for various hydrologic models. Obtaining accurate and reliable precipitation data is thus very important for local, regional, and global hydrologic prediction and water resources management. In this study, the ARPS overestimates precipitation in terms of amplitude based on standard measures and a spatial verification, which causes the difference between distribution function of the ARPS forecasts and observations. The amplitude correction is performed by matching percentile values of the ARPS precipitation forecastsy
k
NWP and observations O
k [30, 31]. All nonzero ARPS precipitation forecasts and observations from calibration data set were sorted separately in ascending order. The percentile values of ARPS forecasts Y
i and observations O
i,  i
=
1
,
…
,
101, for percentiles 0.01,1
,
2
,
…
,
99 and 99.9 of y
k
NWP and O
k were calculated and saved in tables. The corrected precipitation forecast y
k
Cor was obtained using the following:(1)
y
k
Cor
=
y
k
NWP
i
f
y
k
NWP
<
Y
1
y
k
Cor
=
y
k
NWP
+
O
101
-
Y
101
e
l
s
e
i
f
y
k
NWP
>
Y
101
y
k
Cor
=
O
i
+
O
i
+
1
-
O
i
Y
i
+
1
-
Y
i
y
k
NWP
-
Y
i
e
l
s
e
i
f
Y
i
<
y
k
NWP
<
Y
i
+
1
.Taking into account that amplitude error is a function with lead time, the correction parameters are derived for each lead time. The corrected precipitation forecasts were evaluated and compared with the original ARPS forecasts over the five heavy rain events in the summers of 2014 and 2015.


## 5. Verification Methods of Forecasts

Because convective precipitation fields change quickly with time and space, it is difficult to evaluate the convective precipitation forecasts using uniform verification method [32]. We applied standard methods and a spatial verification method in this study. Among the standard methods, we used Bias (B
s), agreement index (d), mean absolute error (MAE), and root mean square error (RMSE) to quantitatively evaluate precipitation forecasts based on grids. The bias is an important measure for hydrologic applications. The agreement index, instead of correlation coefficient, was used to measure the agreement between forecasts and observations because the correlation coefficient has the disadvantage of not being sensitive to linear differences of observation and prediction [31]. MAE and RMSE can quantitatively evaluate forecast error. B
s, d, MAE, and RMSE are computed from the following formulas: (2)
B
s
=
∑
i
=
1
N
F
i
∑
i
=
1
N
O
i
d
=
1
-
∑
i
=
1
N
O
i
-
F
i
2
∑
i
=
1
N
O
i
-
O
-
+
F
i
-
O
-
2
M
A
E
=
1
N
∑
i
=
1
N
O
i
-
F
i
R
M
S
E
=
∑
i
=
1
N
O
i
-
F
i
2
N
,where F
i and O
i are the predicted and observed rainfall at the ith grid point, the number of observations and forecasts is N, and the bar indicates the mean value. A perfect forecast means predicted rainfall field is the same as observation field and would result in B
s
=
1, d
=
1, M
A
E
=
0, and M
A
E
=
0. Although being easily performed, the standard methods have the problem known as the “double penalty” for those precipitation fields with complex structures [31, 33].To avoid the “double penalty,” spatial verification methods, which can identify the sources of forecasts error, have been applied to evaluate high resolution NWP forecasts of precipitation in the last decades. We used the Structure-Amplitude-Location (SAL) score as the supplement for the standard measures in this study. The SAL is an object-based measure method, which considers three components of the structure (S), amplitude (A), and location (L) of precipitation field [33]. The amplitude component A measures the mean precipitation difference over the considered domain between forecasts and observations. The location component L combines information about the distance between the centers of mass of the predicted and observed precipitation of the total field and about the mean displacement of the precipitation objects from the center of mass of the total precipitations field. The structure component S compares the volume of predicted and observed precipitation fields. Positive (negative) value of S indicates widespread (sharp) predicted precipitation fields compared to the observed ones. A perfect forecast would result in S
=
0, A
=
0, and L
=
0.


## 6. Results and Discussion

The ARPS model has been operated in Jiangsu Observatory since 2014. In this study, three precipitation events during the summer of 2014 were used to develop the ARPS model correction parameters, and five precipitation events during the summers of 2014 and 2015 were used to evaluate the original and corrected precipitation forecasts. The evaluation was performed up to 6 h lead times with a spatial resolution of 3 km × 3 km and 1 h intervals.Figure3 shows an example of the ARPS original and corrected hourly precipitation forecasts at lead times from 1 to 6 h with 1 h intervals at a base time of 1500 BJT (Beijing Time) on June 24, 2015, and corresponding radar-based QPEs. In general, the original ARPS precipitation forecasts (Figure 3(a)) overestimate precipitation rate and precipitation extension, while with perfect position of rain band. The overestimation is the most significant at the lead time of 1 h and reduces with increasing lead time. The amplitude correction scheme successfully reduced the amplitude error of the original ARPS forecasts and produced the corrected precipitation forecasts (i.e., Figure 3(b)) similar to observations.Figure 3
Comparison of forecasted precipitation by the original (a) and corrected (b) ARPS initialized at 1500 BJT on 24 June 2015 from 1 h to 6 h lead times with corresponding radar observations (c).The performance of the original ARPS precipitation forecast and effectiveness of the amplitude correction scheme were quantitatively evaluated with agreement index, bias, MAE, and RMSE. Figure4 shows the comparison of the performances between the original and corrected ARPS precipitation forecasts up to the lead time of 6 h with 1 h interval initialized at 1500 BJT on 24 June, 2015. The original ARPS forecast has lower agreement index and higher bias, MAE and RMSE at the lead time of 1 h, which indicates the ARPS may not produce optimal forecasts for very short lead time. The forecast accuracy increases with lead time. The amplitude correction scheme substantially improves the ARPS precipitation forecasts for all the lead times in terms of the considered performance indices of the standard methods. Particularly, the improvement is significant at the lead time of 1 h.Figure 4
Quantitatively compare corrected ARPS precipitation forecasts with the original ARPS precipitation forecasts initialized at 1500 BJT on 24 June 2015.Table1 shows quantitative results of the original and corrected ARPS forecasts for lead times of 1–6 h and a comparison of their outputs by standard measures for each event. In general, the correction scheme improves the forecast performances based on standard measurements for all lead times and each event, especially at 1 h lead time.Table 1
Mean forecast performances based on standard measurements of the original and the corrected ARPS for lead times from 1 h to 6 h for five heavy rain events.
d
Event 1
Event 2
Event 3
Event 4
Event 5
ARPS/Corrected-ARPS
ARPS/Corrected-ARPS
ARPS/Corrected-ARPS
ARPS/Corrected-ARPS
ARPS/Corrected-ARPS
1 h
0.28/0.44
0.29/0.43
0.28/0.51
0.27/0.53
0.10/0.37
2 h
0.28/0.36
0.28/0.37
0.30/0.40
0.36/0.47
0.17/0.27
3 h
0.27/0.35
0.27/0.34
0.28/0.35
0.37/0.43
0.19/0.25
4 h
0.26/0.34
0.24/0.32
0.26/0.29
0.34/0.38
0.18/0.23
5 h
0.29/0.37
0.25/0.29
0.19/0.23
0.32/0.36
0.17/0.21
6 h
0.29/0.36
0.22/0.26
0.19/0.23
0.28/0.31
0.20/0.23
B
s
Event 1
Event 2
Event 3
Event 4
Event 5
ARPS/Corrected-ARPS
ARPS/Corrected-ARPS
ARPS/Corrected-ARPS
ARPS/Corrected-ARPS
ARPS/Corrected-ARPS
1 h
12.15/2.89
10.26/2.89
12.63/2.79
14.91/2.74
12.81/3.15
2 h
6.76/3.23
5.88/2.88
6.75/2.77
6.14/2.61
6.71/3.11
3 h
6.67/3.48
4.39/2.32
4.93/2.33
3.83/1.97
4.50/2.32
4 h
6.19/3.43
4.05/2.42
4.94/2.51
3.36/1.92
3.66/2.06
5 h
5.24/3.26
3.76/2.37
5.14/2.78
3.12/2.02
2.80/1.78
6 h
4.47/2.77
3.40/2.07
4.22/2.26
2.90/1.88
2.08/1.31
MAE (mm)
Event 1
Event 2
Event 3
Event 4
Event 5
ARPS/Corrected-ARPS
ARPS/Corrected-ARPS
ARPS/Corrected-ARPS
ARPS/Corrected-ARPS
ARPS/Corrected-ARPS
1 h
2.26/0.62
2.52/0.76
2.07/0.57
1.41/0.27
1.22/0.31
2 h
1.27/0.70
1.53/0.84
0.96/0.57
0.64/0.30
0.65/0.32
3 h
1.21/0.71
1.42/0.84
0.84/0.58
0.48/0.28
0.45/0.26
4 h
1.20/0.73
1.32/0.91
1.09/0.63
0.45/0.29
0.39/0.24
5 h
1.08/0.73
1.39/0.97
0.88/0.67
0.43/0.30
0.37/0.25
6 h
0.99/0.67
1.36/0.94
0.84/0.63
0.42/0.31
0.34/0.23
RMSE (mm)
Event 1
Event 2
Event 3
Event 4
Event 5
ARPS/Corrected-ARPS
ARPS/Corrected-ARPS
ARPS/Corrected-ARPS
ARPS/Corrected-ARPS
ARPS/Corrected-ARPS
1 h
4.29/1.53
4.83/2.00
3.95/1.48
2.76/0.70
2.97/1.05
2 h
3.12/1.78
3.84/2.22
2.66/1.61
1.47/0.76
1.99/1.10
3 h
3.14/1.80
3.80/2.25
2.57/1.69
1.24/0.76
1.55/0.92
4 h
3.06/1.85
3.56/2.44
3.01/1.76
1.18/0.79
1.39/0.90
5 h
2.79/1.85
3.69/2.53
2.57/1.86
1.10/0.79
1.28/0.93
6 h
2.54/1.63
3.63/2.40
2.44/1.69
1.12/0.84
1.15/0.82To obtain meaningful verification and comparison of the results, average performance indices over five verification heavy precipitation events in the summers of 2014 and 2015 are given in Figure5. As far as the original ARPS forecasts are concerned, the forecast performances except for agreement index improve with increasing lead time. The agreement index changes little over the forecast period. This is the fact that NWP models may not produce optimal predictions at the first short-term due to sensitive to the initial field, spatial resolution, and assimilation data. And the forecast skill improves as they dynamically resolve large-scale flow [10, 34–36].Figure 5
Average forecast performance indices over the considered three heavy precipitation events for lead times from 1 h to 6 h.The amplitude correction scheme shows significant improvement in the original ARPS forecasts in terms of the considered performance indices. Especially at the 1 h ahead, the agreement index is improved from 0.24 to 0.46, the bias for the original ARPS forecast is 12.55, and that is reduced to 2.89 by the amplitude correction scheme. RMSE for the original and corrected ARPS forecasts are 3.67 and 1.35, respectively. The performance of the corrected ARPS forecast changes little over the lead times. It seems that the amplitude correction scheme covered the shortage of NWP models not producing optimal forecast for very short lead times.Evaluation of the original and corrected ARPS forecasts by SAL based on the three considered heavy rain events is shown in Figure6. Most the original ARPS forecasts are characterized by positive A-component values for all the considered lead times, indicating an overestimation of the original ARPS precipitation forecast. Especially for the lead time of 1 h, a cluster of dots are found in the top hand corner of Figure 6(a), which indicates that the ARPS produced very widespread precipitation objects and significantly overestimated precipitation rate. With the increasing lead time, some forecasts can be seen in the second quadrant of the diagram, which implies that the ARPS forecasted overestimated precipitation, with very small and/or too peak objects. Most of the original middle ARPS forecasts are indicated by the red and purple dots at the 1 h lead time, meaning a high-quality forecast in terms of the location of predicted precipitation field. The location of predicted precipitation field is slightly getting poor with the increasing lead time. As shown in Table 1, both the mean values of S-component and A-component move towards centerline with lead time, whereas the mean value of L-component slightly departs from centerline with lead time.Figure 6
SAL diagrams for the hourly precipitation forecasts of the original (a) and the corrected (b) ARPS for lead times from 1 h to 6 h. Every circle indicates three components of SAL for a forecast. The L component is shown by the color of the circles.For the corrected ARPS forecast, at the 1 h lead time, most forecasts are found in the first and second quadrant of the diagram. In the first quadrant, both A-component and S-component of SAL are overestimated. In the second quadrant, forecasts overestimate A-component, whereas underestimate S-component. Compared to the original ARPS forecast, the overestimation of precipitation account is significantly improved. As the increasing lead time, the high density of dots move towards the centerline of the diagram, indicating the overestimation is gradually improving.In general, as shown in Table2, the mean A-components and S-components of the corrected ARPS forecast move towards the centers of the diagram, and the mean L-components are almost unchanged (compared to the original ARPS). The amplitude correction scheme improves the original ARPS precipitation forecast on both the amplitude and structure of precipitation.Table 2
Mean values of S-component, A-component, and L-component of the original and corrected ARPS over three heavy rain events.
Lead times
S (ARPS)
S (Corrected-ARPS)
A (ARPS)
A (Corrected-ARPS)
L (ARPS)
L (Corrected-ARPS)
1 h
1.36
0.08
1.58
0.73
0.22
0.20
2 h
0.70
0.16
1.26
0.71
0.24
0.25
3 h
0.59
0.24
1.05
0.53
0.28
0.28
4 h
0.64
0.25
0.97
0.50
0.30
0.30
5 h
0.47
0.23
0.81
0.43
0.30
0.31
6 h
0.24
0.14
0.66
0.26
0.33
0.34


## 7. Summary and Conclusions

This paper quantitatively evaluated the original ARPS precipitation forecasts and addressed and corrected the forecast error in Jiangsu. The forecast performances were evaluated and compared as a function of lead times of 1 h to 6 h using standard measures and a spatial verification method.Even with atmospheric dynamic constraints and data assimilation techniques, the ARPS may not produce optimal forecasts for very short lead times due to its sensitivity to the initial field. In general, the ARPS model yields overestimated and widespread precipitation at a lead time of 1 h, which was confirmed by the significantly large bias and that most forecasts were concentrated in the top hand corner of Figure6(a). The forecasting skill gradually improves with lead time; however the ARPS model overestimates precipitation at all of the considered lead times.The amplitude correction scheme based on distribution function matching methods successfully improved the original ARPS precipitation forecasts, which can be confirmed by the considered performances indices of standard measures and both mean A-component and S-component of SAL. Especially at the lead time of 1 h, the amplitude correction scheme significantly reduces the forecast errors. It seems that the amplitude correction scheme effectively overcomes the problem of the ARPS sensitive to the initial field, resulting in the forecast skill of the corrected ARPS changing little with increasing lead time.The applicability and portability of the correction methods depend on the NWP forecasts and local precipitation types. The correction methods should be recalibrated if the methods are used for other area.



---

*Source: 8476720-2016-08-11.xml*