# Performance of Three Reanalysis Precipitation Datasets over the Qinling-Daba Mountains, Eastern Fringe of Tibetan Plateau, China

Authors: Gefei Wang; Xiaowen Zhang; Shiqiang Zhang
Journal: Advances in Meteorology
 (2019)

---

## Abstract

Evaluation of different reanalysis precipitation datasets is of great importance to understanding the hydrological processes and water resource management practice in the Qinling-Daba Mountains (QDM), located at the eastern fringe of the Tibetan Plateau. Although the evaluation of satellite precipitation data in this region has been performed, another kind of popular precipitation product-reanalysis dataset has not been assessed in depth. Three popular reanalysis precipitation datasets, including ERA-Interim Reanalysis of European Centre for Medium Forecasts (ERA-Interim), Japanese 55-year Reanalysis (JRA-55), and National Centers for Environmental Prediction/National Center for Atmospheric Research Reanalysis-1 (NCEP/NCAR-1) were evaluated against rain gauge data over the Qinling-Daba Mountains from 2000 to 2014 on monthly, seasonal, and annual scales. Different statistical measures based on the Correlation Coefficient (CC), relative BIAS (BIAS), Root-Mean-Square Error (RMSE), and Mean Absolute Error (MAE) were adopted to determine the performance of the above reanalysis datasets. Results show that ERA-Interim and JRA-55 have good performance on a monthly scale and annual scale. However, the NCEP/NCAR-1 has the least BIAS with the observed precipitation in annual scale in QDM. All reanalysis datasets performed better in spring, summer, and autumn than in winter. The advantages of involving more precipitation observation stations was probably the main reason of the different performance of three precipitation reanalysis products, and the benefit of a four-dimensional variational analysis model over a three-dimensional variational analysis model may be another reason. The evaluation suggested that ERA-Interim is more suitable for study the precipitation and water cycles in the QDM.


---

## Body

## 1. Introduction

As a major component of the hydrological and energy cycle, the spatial and temporal patterns of precipitation greatly impact land surface hydrological fluxes and states [1–3]. Precipitation has a number of applications in various disciplines and studies, such as hydrology and water cycle processes, snowfall estimation, and climate research [4]; thus, its accuracy is pertinent. Traditionally, surface observations from rain gauges are regarded as one of the most accurate measurements for precipitation at a point scale. However, surface observation networks are sparse in many developing countries [5] due to the high cost of establishing and maintaining infrastructure [6]. Precise and continuous information on precipitation remains a challenging task, especially in remote mountainous areas.Over the past several decades, tremendous efforts have been made to measure and monitor precipitation [7], which facilitate and promote the development of numerous global and quasiglobal precipitation products, including satellite-based datasets and reanalysis-based products using input sources, such as ground-based observations and satellite estimates. Advantages offered by satellites encourage the precipitation retrievals through visible spectrometry (VIS)/infrared (IR), passive microwave, active microwave, and multisensors methods, which are the foundation of satellite-based precipitation estimates [4]. Conversely, various observational data and numerical weather prediction products are fused and integrated by data assimilation systems to produce the reanalysis datasets [8]. Satellite-based estimates and reanalysis datasets both need to be evaluated against in situ observations and calibrated before being implemented into various applications [9].Therefore, various research studies have been conducted to explore the performance of the different datasets on regional [10–14] and global scales [15, 16]. Many studies have found that the datasets agree well on large scales but exhibit obvious and marked differences in various regions. For instance, Janowiak et al. [17] compared National Centers for Environmental Prediction (NCEP)-National Center for Atmospheric Research (NCAR) reanalysis data and Global Precipitation Climatology Project (GPCP) rain gauge-satellite combined dataset globally over the period 1988–1995 and found good agreement between large-scale patterns but poor performance in some regional characteristics, such as oceans and equatorial land regions. Huffman et al. [18] and Kummerow et al. [19] explained the necessities for adequate validations on regional scales instead of using global approaches. Taking into account the above reasons, although Zhao and Fu [20], Ma et al. [21], and other researchers [22–24] have performed relevant studies in China, conclusions are still unclear on a regional scale. Therefore, it is necessary to validate and evaluate the performance of various precipitation datasets in certain areas, especially in mountainous areas.The Qinling-Daba Mountains (QDM), which geographically and climatologically divide northern and southern China with the Huaihe River, serve as an important water source for the middle route of South-to-North Water Diversion Project in China. Meanwhile, the QDMs are located at the eastern fringe of the mountain region of the Qinghai Tibetan Plateau, which is the source of many large rivers and called “Asia Water Tower.” Knowledge of precipitation in the QDM is of great significance to water resources management, hydrological modeling, and climate research in the immediate and surrounding regions. Given the large variations in the terrain, mountain systems develop considerably complex local and regional climate systems [25], which increase the difficulties in obtaining accurate precipitation information.The development of precipitation datasets provides beneficial conditions to measure precipitation in the QDM. Ren et al. [26] evaluated the precipitation from the Tropical Rainfall Measuring Mission (TRMM 3B42) in the surrounding area. Wang et al. [27] assessed the performance of the Climate Prediction Center morphing technique (CMORPH), Global Precipitation Climatology Project (GPCP-2), TRMM 3B43, Global Precipitation Climatology Center (GPCC), and China Meteorological Forcing Data developed by the Institute of Tibetan Plateau Research, Chinese Academy of Sciences (ITPCAS) over the QDM and reclaimed that ITPCAS and TRMM 3B43 performed better overall. These results concentrated on satellite-based precipitation datasets. Another kind of precipitation product, reanalysis-based datasets, which integrate ground observation data and atmospheric models, provide an alternative perspective for understanding the spatial distribution of precipitation in mountainous areas where satellite data are difficult to cove or large errors exist. The reanalysis precipitation products have been commonly used in climatology and basin hydrometeorology [28–31], but have not been fully evaluated in the QDM.The main objective of this work is to evaluate three reanalysis-based datasets on monthly, seasonal, and annual scales through observations in the QDM during 2000–2014. This paper is organized as follows: Section2 describes the basic information of the study area; Section 3 introduces the relevant datasets and evaluation methods; Section 4 presents the evaluation results; Section 5 discusses the probable reason behind the different performances of the reanalysis datasets; and Section 6 summarizes the conclusions.


## 2. Study Area

The QDM, referring to both Qinling Mountain and Daba Mountain, are located in central China with an area of about 222,300 km2, from 30°50′ N to 34°59′ N latitude and from 102°54′ E to 112°40′ E longitude (Figure 1) [32, 33]. This area is roughly consistent with 0°C isothermal contours in January, 800 mm isohyet curves, and annual 2000 h sunshine hour contour lines in China [34, 35]. The elevation in the QDM varies greatly, where the difference between the maximum and minimum elevation is almost 4000 m (Figure 1), and precipitation distributed with significant spatial and temporal heterogeneity. The main land cover of QDM is forest and shrub, where forest is mainly distributed in west of QDM and shrub mainly distributed in east of QDM. The soil texture has vertical zonal distribution from yellow clunamon soil at the foot of the mountains to mountain dark brown soil at the peak of the mountains. The spatial distribution of average of annual precipitation during 1958 to 2014 shows that the annual precipitation in the northwestern region is about 500 mm, which is far less than that in the southwestern areas (about 1300 mm) (Figure 2(a)). Meanwhile, the main seasonal pattern of rainfall suggests that the QDM have four distinct seasons, and precipitation mainly occurs during warmer months from May to September (Figure 2(b)).Figure 1
The location and the spatial distribution of the rain gauge stations over the Qinling-Daba Mountains in China (Figure1 is reproduced from Wang et al. [27] under the Creative Commons Attribution License/public domain).Figure 2
(a) The spatial distribution of annual rainfall; (b) the time distribution of monthly rainfall and air temperature throughout the year over the Qinling-Daba Mountains, China (Figure2 is reproduced from Wang et al. [27], under the Creative Commons Attribution License/public domain).


## 3. Data and Methodology

### 3.1. Reanalysis Precipitation Datasets

ERA-Interim, JRA-55, and NCEP/NCAR-1, which are popular used in many studies [17, 21, 28–31] and are available to public, were chosen as the reanalysis dataset to be evaluated in this study.ERA-Interim is a global reanalysis product created by European Center for Medium-Range Weather Forecasts (ECMWF) [36], which was initiated in 1979. This reanalysis dataset is carried out with a 4-Dimensional VARiational analysis (4D-VAR) data assimilation scheme, a better formulation of background error constraint, a new humidity analysis, and many other improvements, to address several difficult data assimilation problems encountered during the production of ERA-40 and achieve great progresses compared to ERA-40 [37]. The overwhelming majority of observation data, and most of the increase over time, originate from satellites. Such data include clear-sky radiance measurements from polar-orbiting and geostationary sounders and imagers, atmospheric motion vectors derived from geostationary satellites, scatterometer wind data, and ozone retrievals from various satellite-borne sensors. The total precipitable vapor estimates are also derived from satellite observations. Although manual and automatic ground observations of precipitation were also considered, the number of stations in the QDM involved in data assimilation is unclear. In this study, monthly ERA-Interim was obtained from ECMWF on a fixed grid of 0.75° × 0.75° (http://apps.ecmwf.int/).JRA-55 is a global reanalysis dataset constructed by the Japan Meteorological Agency (JMA) [38]. JRA-55 employs a 4D-VAR with variational bias correction for satellite radiances. It aims at providing a comprehensive atmospheric dataset that is suitable for studies on climate change and related issues [39]. The observation data primarily included conventional data (such as tropical cyclone wind retrievals, pilot balloons, wind profilers, etc.), wind data retrieved from geostationary TOVS (TIROS Operational Vertical Sounder), ATOVS (advanced TOVS), AMV (Atmospheric Motion Vector), CSR (Clear-Sky Radiance) data, and other remote sensing data [40]. Moreover, newly available observational datasets were collected and used whenever possible [41]. In this study, monthly data from JRA-55 were adopted, which are accessed for free online (http://jra.kishou.go.jp/).NCEP/NCAR-1 is a global reanalysis dataset of atmosphere fields produced by the National Centers for Environmental Prediction and National Center for Atmospheric Research to meet the needs of research and climate monitoring communities [42]. A 3D-VAR (three-dimensional variational analysis) is used in the assimilation system of NCEP/NCAR-1. The horizontal resolution is T62 Gaussian grid with 192 × 94 grids of the overall dataset [8]. This model includes parametrizations of all major physical processes, such as convection, clouds, and an interactive surface hydrological model on a global scale. The NCEP/NCAR-1 precipitation field from short-range model forecast accumulations, but observed precipitation is not used in the assimilation phase of the model [43]. Monthly data from NCEP/NCAR-1 were used in this work, which can be downloaded online (https://www.esrl.noaa.gov/).


### 3.2. Gauged Precipitation Data

The China Meteorological Administration (CMA) provided daily in situ observational precipitation data over the QDM during 2000–2014. The precipitation is manually observed at 8 : 00 and 20 : 00 per day by a rain gauges without windproof fences, the area of the collector orifice is 200 cm2, ground stations using the same criteria of CMA in the observation field with 25 m × 25 m with short grass cover, the gauge orifice is 0.7 m above surface. Snow collected in precipitation gauges was melted after each observation and then measured using a standard glass graduated measuring cylinder. Routine maintenance includes the gauge and the field [44].A 0.1° buffer both in latitude and longitude direction of the study area boundary was utilized to ensure that the 27 chosen rain gauge stations were able to delineate relatively accurate spatial distribution of precipitation as much as possible [27]; 18 stations have continuous precipitation data from 2000–2014, while the remaining 9 stations do not. Therefore, 27 meteorological stations were used in the overall evaluation of monthly and annual precipitation, and 18 stations were used for circumstances requiring continuous time series. Manual quality control was carried out, and the gauged precipitation data were used as the reference data to evaluate the performance of reanalysis precipitation datasets in the study. Information and record periods of each gauge station are listed in Table 1.Table 1
The basic information of gauge station over the Qinling-Daba Mountains, China.
Station name
Latitude (°)
Longitude (°)
Elevation (m a.s.l.)
Observation years
Minxian
34.43
104.02
2315
2000–2014
Wudu
33.4
104.92
1079
2000–2014
Tianshui
34.58
105.75
1142
2000–2003, 2007–2008
Beidao
34.57
105.87
1085
2004–2014
Baoji
34.35
107.13
612
2000–2004, 2007–2008
Huashan
34.48
110.08
2065
2000–2014
Lushi
34.05
111.03
569
2000–2014
Luanchuan
33.78
111.6
750
2000–2014
Lueyang
33.32
106.15
794
2000–2014
Liuba
33.65
106.95
1547
2009–2014
Hanzhong
33.07
107.03
510
2000–2014
Foping
33.52
107.98
827
2000–2014
Shangzhou
33.87
109.97
742
2000–2014
Zhen’an
33.43
109.15
694
2000–2014
Shangnan
33.46
110.58
1137
2009–2014
Xixia
33.3
111.5
250
2000–2014
Guangyuan
32.43
105.85
514
2000–2014
Ningqiang
32.84
105.95
1400
2009–2014
Shiquan
33.05
108.27
485
2000–2014
Wanyuan
32.07
108.03
674
2000–2014
Zhenba
32.56
107.91
1231
2009–2014
Ankang
32.72
109.03
291
2000–2014
Yunxi
33
110.42
249
2000–2014
Yunxian
32.85
110.82
202
2007–2008
Fangxian
32.03
110.77
427
2000–2014
Zhenping
31.91
109.51
1615
2009–2014
Fengjie
31.02
109.53
300
2000–2014


### 3.3. Evaluation Method

In terms of temporal resolution, daily gauged precipitation data were accumulated to monthly and annual data. Meanwhile, the precipitation rate data from JRA-55 and NCEP/NCAR-1 were multiplied by corresponding times to obtain rainfall amount which is on the same time scale as gauged rainfall. The monthly total precipitation data of ERA-Interim were obtained by accumulating the daily precipitation. The seasonal total precipitation were summed from monthly precipitation, including winter precipitation (December, January, and February), spring precipitation (March, April, and May), summer precipitation (June, July, and August), and autumn precipitation (September, October, and November).It is common practice in evaluation studies to compare the point-based rain gauge data against the grid-based precipitation datasets. Given the 18 stations with continuous time series over the QDM, a point-pixel comparison was performed in this study to avoid errors by gridding the rain gauge data [11, 45]. The rainfall values from each rain gauge and the grid where the same gauge is located were extracted in pairs for evaluation. Before that, ERA-Interim and JRA-55 were resampled to horizontal 0.5° × 0.5° grid scales to acquire a uniform spatial resolution by bilinear interpolation, which is a popular method in meteorology and climate studies [46, 47]. However, considering the unequal spacing between x and y coordinates between the grid points of NCEP/NCAR-1, resampling will introduce errors. Therefore, rain gauge data were directly compared against the nearest grid points of NCEP/NCAR-1 in the original resolution without resampling.To quantitatively assess the performance of ERA-Interim, JRA-55, and NCEP/NCAR-1 in the QDM, the following several statistical indices were obtained and compared:(1)CC=∑i=1nGi−G¯¯Pi−P¯¯∑i=1nGi−G¯¯2Pi−P¯¯2,BIAS=∑i=1nGi−Pi∑i=1nGi×100%,RMSE=1n∑i=1nGi−Pi2,MAE=1n∑i=1nGi−Pi,where n is the total number of samples; Gi represents the gauged precipitation at ith month; Pi is the precipitation at ith month from reanalysis precipitation datasets; and G¯¯ and P¯¯ are the average values over n months of Gi and Pi, respectively. The Pearson correlation coefficient (CC) has no units and is used to assess the degree of agreement that reflects the level of linear correlation, which varies from −1 to 1, where positive and negative values indicate positive and negative correlation, respectively. The relative bias (BIAS) provides information on the magnitude of underestimation or overestimation between two datasets, in which the closer to 0 the BIAS is, the better performance the precipitation dataset has. The root-mean-square error (RMSE) is sensitive to the maximum and minimum values, and the mean absolute error (MAE) demonstrates the magnitude of mean error. The different indices all used in conjunction to determine the performance of the reanalysis precipitation datasets.Given the emphasis in multiple related studies on assessment of precipitation products [9, 48], CC is regarded as the primary and principle indicator to evaluate the accuracy of precipitation products at various circumstances. The T-test is also performed to verify the statistical significance of CC in advance [49, 50] after checking if the data are normally distributed. BIAS is used to determine the scale of underestimation or overestimation for true precipitation, while RMSE and MAE are utilized to measure the specific errors of the precipitation products.The absolute precipitation differences (PD) and percentage of PD (PPD) were adopted as two different methods to quantitatively determine the agreement between precipitation datasets and gauge data during the dry and wet years:(2)PD=estimated−observed,PPD=estimated−observedobserved×100%.


### 3.4. Precipitation Centroid

The spatial pattern of the precipitation and its temporal change is one of the characteristics of regional precipitation and hydrological process. Traditional spatial evaluation of precipitation products directly compares the spatial distribution of rainfall, which lack quantitative descriptions. Therefore, this study implemented precipitation centroid movement over a 15-year period from 2000 to 2014 to further explore the effectiveness of ERA-Interim, JRA-55, and NCEP/NCAR-1.A centroid, which stems from the concept of center of mass (or gravity) in physics, was first introduced in humanities and social fields, such as population, economy, and tourism [51–54] and has recently been applied to measure precipitation spatial heterogeneity [55, 56]. The precipitation centroid is defined as the point where the moment of precipitation reaches balance in the space plane of the study area that reflects the spatial distribution of precipitation [48], which is introduced to further determine the difference between gauge observation and ERA-Interim, JRA-55, and NCEP/NCAR-1. The moving trajectory of the precipitation centroid displays the change in the spatial distribution of precipitation, which means the similarity between precipitation from products and gauges can be used to verify their spatial heterogeneity agreement [55, 56]. Furthermore, the distance between precipitation centroids of two adjacent years is regarded as the migration distance, and the total migration distances are compared to quantitatively measure the precision of precipitation datasets. We use the migration distance between adjacent years to quantitatively determine the performance of the three reanalysis precipitation datasets.Coordinates of the precipitation centroid were calculated using the following formulas:(3)X=∑i=1nPixi∑i=1nPi,Y=∑i=1nPiyi∑i=1nPi,where n is the number of the rain gauge stations; the location of a rain gauge is (xi,yi); and P represents the amount of precipitation from observation and precipitation products.


## 3.1. Reanalysis Precipitation Datasets

ERA-Interim, JRA-55, and NCEP/NCAR-1, which are popular used in many studies [17, 21, 28–31] and are available to public, were chosen as the reanalysis dataset to be evaluated in this study.ERA-Interim is a global reanalysis product created by European Center for Medium-Range Weather Forecasts (ECMWF) [36], which was initiated in 1979. This reanalysis dataset is carried out with a 4-Dimensional VARiational analysis (4D-VAR) data assimilation scheme, a better formulation of background error constraint, a new humidity analysis, and many other improvements, to address several difficult data assimilation problems encountered during the production of ERA-40 and achieve great progresses compared to ERA-40 [37]. The overwhelming majority of observation data, and most of the increase over time, originate from satellites. Such data include clear-sky radiance measurements from polar-orbiting and geostationary sounders and imagers, atmospheric motion vectors derived from geostationary satellites, scatterometer wind data, and ozone retrievals from various satellite-borne sensors. The total precipitable vapor estimates are also derived from satellite observations. Although manual and automatic ground observations of precipitation were also considered, the number of stations in the QDM involved in data assimilation is unclear. In this study, monthly ERA-Interim was obtained from ECMWF on a fixed grid of 0.75° × 0.75° (http://apps.ecmwf.int/).JRA-55 is a global reanalysis dataset constructed by the Japan Meteorological Agency (JMA) [38]. JRA-55 employs a 4D-VAR with variational bias correction for satellite radiances. It aims at providing a comprehensive atmospheric dataset that is suitable for studies on climate change and related issues [39]. The observation data primarily included conventional data (such as tropical cyclone wind retrievals, pilot balloons, wind profilers, etc.), wind data retrieved from geostationary TOVS (TIROS Operational Vertical Sounder), ATOVS (advanced TOVS), AMV (Atmospheric Motion Vector), CSR (Clear-Sky Radiance) data, and other remote sensing data [40]. Moreover, newly available observational datasets were collected and used whenever possible [41]. In this study, monthly data from JRA-55 were adopted, which are accessed for free online (http://jra.kishou.go.jp/).NCEP/NCAR-1 is a global reanalysis dataset of atmosphere fields produced by the National Centers for Environmental Prediction and National Center for Atmospheric Research to meet the needs of research and climate monitoring communities [42]. A 3D-VAR (three-dimensional variational analysis) is used in the assimilation system of NCEP/NCAR-1. The horizontal resolution is T62 Gaussian grid with 192 × 94 grids of the overall dataset [8]. This model includes parametrizations of all major physical processes, such as convection, clouds, and an interactive surface hydrological model on a global scale. The NCEP/NCAR-1 precipitation field from short-range model forecast accumulations, but observed precipitation is not used in the assimilation phase of the model [43]. Monthly data from NCEP/NCAR-1 were used in this work, which can be downloaded online (https://www.esrl.noaa.gov/).


## 3.2. Gauged Precipitation Data

The China Meteorological Administration (CMA) provided daily in situ observational precipitation data over the QDM during 2000–2014. The precipitation is manually observed at 8 : 00 and 20 : 00 per day by a rain gauges without windproof fences, the area of the collector orifice is 200 cm2, ground stations using the same criteria of CMA in the observation field with 25 m × 25 m with short grass cover, the gauge orifice is 0.7 m above surface. Snow collected in precipitation gauges was melted after each observation and then measured using a standard glass graduated measuring cylinder. Routine maintenance includes the gauge and the field [44].A 0.1° buffer both in latitude and longitude direction of the study area boundary was utilized to ensure that the 27 chosen rain gauge stations were able to delineate relatively accurate spatial distribution of precipitation as much as possible [27]; 18 stations have continuous precipitation data from 2000–2014, while the remaining 9 stations do not. Therefore, 27 meteorological stations were used in the overall evaluation of monthly and annual precipitation, and 18 stations were used for circumstances requiring continuous time series. Manual quality control was carried out, and the gauged precipitation data were used as the reference data to evaluate the performance of reanalysis precipitation datasets in the study. Information and record periods of each gauge station are listed in Table 1.Table 1
The basic information of gauge station over the Qinling-Daba Mountains, China.
Station name
Latitude (°)
Longitude (°)
Elevation (m a.s.l.)
Observation years
Minxian
34.43
104.02
2315
2000–2014
Wudu
33.4
104.92
1079
2000–2014
Tianshui
34.58
105.75
1142
2000–2003, 2007–2008
Beidao
34.57
105.87
1085
2004–2014
Baoji
34.35
107.13
612
2000–2004, 2007–2008
Huashan
34.48
110.08
2065
2000–2014
Lushi
34.05
111.03
569
2000–2014
Luanchuan
33.78
111.6
750
2000–2014
Lueyang
33.32
106.15
794
2000–2014
Liuba
33.65
106.95
1547
2009–2014
Hanzhong
33.07
107.03
510
2000–2014
Foping
33.52
107.98
827
2000–2014
Shangzhou
33.87
109.97
742
2000–2014
Zhen’an
33.43
109.15
694
2000–2014
Shangnan
33.46
110.58
1137
2009–2014
Xixia
33.3
111.5
250
2000–2014
Guangyuan
32.43
105.85
514
2000–2014
Ningqiang
32.84
105.95
1400
2009–2014
Shiquan
33.05
108.27
485
2000–2014
Wanyuan
32.07
108.03
674
2000–2014
Zhenba
32.56
107.91
1231
2009–2014
Ankang
32.72
109.03
291
2000–2014
Yunxi
33
110.42
249
2000–2014
Yunxian
32.85
110.82
202
2007–2008
Fangxian
32.03
110.77
427
2000–2014
Zhenping
31.91
109.51
1615
2009–2014
Fengjie
31.02
109.53
300
2000–2014


## 3.3. Evaluation Method

In terms of temporal resolution, daily gauged precipitation data were accumulated to monthly and annual data. Meanwhile, the precipitation rate data from JRA-55 and NCEP/NCAR-1 were multiplied by corresponding times to obtain rainfall amount which is on the same time scale as gauged rainfall. The monthly total precipitation data of ERA-Interim were obtained by accumulating the daily precipitation. The seasonal total precipitation were summed from monthly precipitation, including winter precipitation (December, January, and February), spring precipitation (March, April, and May), summer precipitation (June, July, and August), and autumn precipitation (September, October, and November).It is common practice in evaluation studies to compare the point-based rain gauge data against the grid-based precipitation datasets. Given the 18 stations with continuous time series over the QDM, a point-pixel comparison was performed in this study to avoid errors by gridding the rain gauge data [11, 45]. The rainfall values from each rain gauge and the grid where the same gauge is located were extracted in pairs for evaluation. Before that, ERA-Interim and JRA-55 were resampled to horizontal 0.5° × 0.5° grid scales to acquire a uniform spatial resolution by bilinear interpolation, which is a popular method in meteorology and climate studies [46, 47]. However, considering the unequal spacing between x and y coordinates between the grid points of NCEP/NCAR-1, resampling will introduce errors. Therefore, rain gauge data were directly compared against the nearest grid points of NCEP/NCAR-1 in the original resolution without resampling.To quantitatively assess the performance of ERA-Interim, JRA-55, and NCEP/NCAR-1 in the QDM, the following several statistical indices were obtained and compared:(1)CC=∑i=1nGi−G¯¯Pi−P¯¯∑i=1nGi−G¯¯2Pi−P¯¯2,BIAS=∑i=1nGi−Pi∑i=1nGi×100%,RMSE=1n∑i=1nGi−Pi2,MAE=1n∑i=1nGi−Pi,where n is the total number of samples; Gi represents the gauged precipitation at ith month; Pi is the precipitation at ith month from reanalysis precipitation datasets; and G¯¯ and P¯¯ are the average values over n months of Gi and Pi, respectively. The Pearson correlation coefficient (CC) has no units and is used to assess the degree of agreement that reflects the level of linear correlation, which varies from −1 to 1, where positive and negative values indicate positive and negative correlation, respectively. The relative bias (BIAS) provides information on the magnitude of underestimation or overestimation between two datasets, in which the closer to 0 the BIAS is, the better performance the precipitation dataset has. The root-mean-square error (RMSE) is sensitive to the maximum and minimum values, and the mean absolute error (MAE) demonstrates the magnitude of mean error. The different indices all used in conjunction to determine the performance of the reanalysis precipitation datasets.Given the emphasis in multiple related studies on assessment of precipitation products [9, 48], CC is regarded as the primary and principle indicator to evaluate the accuracy of precipitation products at various circumstances. The T-test is also performed to verify the statistical significance of CC in advance [49, 50] after checking if the data are normally distributed. BIAS is used to determine the scale of underestimation or overestimation for true precipitation, while RMSE and MAE are utilized to measure the specific errors of the precipitation products.The absolute precipitation differences (PD) and percentage of PD (PPD) were adopted as two different methods to quantitatively determine the agreement between precipitation datasets and gauge data during the dry and wet years:(2)PD=estimated−observed,PPD=estimated−observedobserved×100%.


## 3.4. Precipitation Centroid

The spatial pattern of the precipitation and its temporal change is one of the characteristics of regional precipitation and hydrological process. Traditional spatial evaluation of precipitation products directly compares the spatial distribution of rainfall, which lack quantitative descriptions. Therefore, this study implemented precipitation centroid movement over a 15-year period from 2000 to 2014 to further explore the effectiveness of ERA-Interim, JRA-55, and NCEP/NCAR-1.A centroid, which stems from the concept of center of mass (or gravity) in physics, was first introduced in humanities and social fields, such as population, economy, and tourism [51–54] and has recently been applied to measure precipitation spatial heterogeneity [55, 56]. The precipitation centroid is defined as the point where the moment of precipitation reaches balance in the space plane of the study area that reflects the spatial distribution of precipitation [48], which is introduced to further determine the difference between gauge observation and ERA-Interim, JRA-55, and NCEP/NCAR-1. The moving trajectory of the precipitation centroid displays the change in the spatial distribution of precipitation, which means the similarity between precipitation from products and gauges can be used to verify their spatial heterogeneity agreement [55, 56]. Furthermore, the distance between precipitation centroids of two adjacent years is regarded as the migration distance, and the total migration distances are compared to quantitatively measure the precision of precipitation datasets. We use the migration distance between adjacent years to quantitatively determine the performance of the three reanalysis precipitation datasets.Coordinates of the precipitation centroid were calculated using the following formulas:(3)X=∑i=1nPixi∑i=1nPi,Y=∑i=1nPiyi∑i=1nPi,where n is the number of the rain gauge stations; the location of a rain gauge is (xi,yi); and P represents the amount of precipitation from observation and precipitation products.


## 4. Evaluation Results

The performance of ERA-Interim, JRA-55, and NCEP/NCAR-1 on monthly, seasonal, and annual scales is presented in this section. In this study, the monthly scale was used as the base time scale, and movement of the precipitation centroid was analyzed to further explore the performance of the three datasets over the QDM, China.


### 4.1. Performance on Monthly Scale

Three reanalysis precipitation datasets were first validated on a monthly scale. To eliminate the influence of the seasonal cycle on CC, the CC of each precipitation dataset was calculated per month (Figure3). All datasets passed the significance test at the 99% confidence level except NCEP/NCAR-1 in certain months. It is clear that ERA-Interim outperformed the other reanalysis precipitation products in most months and had an average CC of 0.64. JRA-55 performed second-best with a mean CC of 0.58. Meanwhile, JRA-55 was better than ERA-Interim in June, July, August, and September, and ERA-Interim had higher agreement with observed precipitation than JRA-55 in the remaining eight months. In addition, JRA-55 simulated precipitation better in months with abundant rainfall. NCEP/NCAR-1 performed worst with the lowest CC (about 0.22) among the evaluated precipitation datasets.Figure 3
The line chart of correlation coefficient (CC) between monthly observed precipitation and ERA-Interim, JRA-55, and NCEP/NCAR-1 precipitation during 2000 to 2014 over the Qinling-Daba Mountains, China.Therefore, the average CCs of the twelve months of a specific year were treated as the overall performance of every single precipitation product on a monthly scale, which are shown in Table2 with the other evaluation indices.Table 2
The average of CC, BIAS, RMSE, and MAE between three reanalysis precipitation datasets and gauged precipitation data on a monthly scale during 2000 to 2014 over the Qinling-Daba Mountains, China.
Index
ERA-Interim
JRA-55
NCEP/NCAR-1
CC
0.64
0.58
0.22
BIAS (%)
21.78
16.57
−6.31
RMSE (mm)
48.56
43.05
64.93
MAE (mm)
30.42
30.81
39.18Overall, ERA-Interim and JRA-55 revealed a similar ability to simulate rainfall for evaluation indices. ERA-Interim had better CC and MAE, and JRA-55 had a better BIAS and RMSE. NCEP/NCAR-1 had the lowest CC and largest RMSE and MAE, suggesting that NCEP/NCAR-1 is the poor performing dataset, even though it had a low BIAS.The spatial distribution of evaluation indices at individual gauges was obtained to investigate the performance of the three precipitation products, as illustrated in Figures4 and 5.Figure 4
Correlation coefficient at each precipitation gauge stations over the Qinling-Daba Mountains for monthly precipitation between (a) ERA-Interim and gauges, (b) JRA-55 and gauges, and (c) NCEP/NCAR-1 and gauges.
(a)
(b)
(c)Figure 5
BIAS (%) at each gauge over the Qinling-Daba Mountains for monthly precipitation between (a) ERA-Interim and gauges, (b) JRA-55 and gauges, and (c) NCEP/NCAR-1 and gauges.
(a)
(b)
(c)The spatial distribution of correlation coefficients suggests that CCs at most sites were greater than 0.5 for ERA-Interim and JRA-55 (Figure4). ERA-Interim and JRA-55 have the same number of stations with CC values higher than 0.7. NCEP/NCAR-1 (Figure 4(c)) showed the worst performance with lower CCs at most stations, where five stations even had CC values lower than 0.5.Furthermore, stations with relatively high CCs were concentrated in the northeastern region of the QDM, which due to the relatively low altitude in the eastern region. The influence of terrain in the eastern region is less than the western and northwestern part of QDM. It is well established that the topographic and orographic influences on precipitation formation and propagation. It is expected that there is less precipitation on the leeward side of the mountain on the western side of the QDM because of the dry-adiabatic decent of air, which leads to lower CCs. It is worth noting that ERA-Interim and JRA-55 had the lowest CC at Wudu station at the same time.ERA-Interim and JRA-55 had similar distributions of BIAS, which are both overestimated the precipitation at the most stations. Comparatively, NCEP/NCAR-1 underestimated rainfall. ERA-Interim, JRA-55, and NCEP/NCAR-1 all had the largest errors at Wudu station with positive BIAS values of 130.4%, 123.3%, and 223.4%, respectively, which may be attributed to the complex terrain in Bailongjiang Valley. Another interesting phenomenon was the underestimation of precipitation at Huashan station for ERA-Interim, JRA-55, and NCEP/NCAR-1. Considering that the elevation of Huashan station is 2054 m a.s.l., the large wind at such a high elevation leads to lower precipitation gauge capture rate [57], and precipitation of reanalysis products is expected to be overestimated. However, it is difficult to determine the contribution of the underestimation of observed precipitation and the overestimation of the reanalysis products, respectively.


### 4.2. Performance on Seasonal Scale

In this study, precipitation was greater during summer (June–August) and autumn (September–November) than in spring (March–May) and winter (December–next February) (Figure2(b)). To understand the seasonal pattern of errors for reanalysis precipitation datasets comprehensively, the overall performance of ERA-Interim, JRA-55, and NCEP/NCAR-1 were evaluated and are shown in Table 3.Table 3
CC, BIAS, RMSE, and MAE between three reanalysis precipitation datasets and gauged precipitation data at the seasonal scale over the Qinling-Daba Mountains, China (all products passed the significance test at the 99% confidence level).
Season
Dataset
CC
BIAS (%)
RMSE (mm)
MAE (mm)
Spring
ERA-Interim
0.67
23.61
69.45
55.79
JRA-55
0.51
35.61
89.65
71.88
NCEP/NCAR-1
0.07
−5.55
120.12
88.11
Summer
ERA-Interim
0.57
27.80
187.92
154.01
JRA-55
0.66
1.86
120.62
95.39
NCEP/NCAR-1
0.12
1.99
211.08
170.28
Autumn
ERA-Interim
0.66
10.38
87.10
67.36
JRA-55
0.70
8.49
83.21
62.20
NCEP/NCAR-1
0.05
−14.92
171.11
128.61
Winter
ERA-Interim
0.49
109.77
36.03
31.26
JRA-55
0.17
209.67
65.95
58.16
NCEP/NCAR-1
0.05
16.03
29.43
21.06All reanalysis datasets displayed higher CC values in spring, summer, and autumn than winter and had lower BIAS values in summer and autumn than spring and winter. Thus, precipitation datasets performed better in warmer and wetter seasons (summer and autumn), which may be conducive to monitoring and predicting geologic hazards caused by heavy rain in a short time period in the QDM. The larger errors in RMSE and MAE in summer and autumn may be due to the fact that rainfall concentrated during those seasons over the QDM (Figure2).It is worth mentioning that JRA-55 coincided worse performing with observed rainfall in spring and winter but performed better in summer and autumn than ERA-Interim, which indicates that it may be better to use JRA-55 for simulating abundant precipitation than ERA-Interim over the QDM. The CCs of NCEP/NCAR-1 were too low to simulate true rainfall, making NCEP/NCAR-1 the worst performance dataset of the three.In summary, all evaluated datasets displayed higher accuracy in summer and autumn than spring and, especially winter, when the performance of the datasets was much worse than the other seasons. ERA-Interim and JRA-55 had similar performance and good agreement with observed precipitation, while NCEP/NCAR-1 showing the poorest performance.


### 4.3. Performance on Annual Scale

The average annual precipitation of each dataset was calculated and compared with the in situ observed precipitation on an annual scale (Figure6).Figure 6
The line chart of regional average annual precipitation from rain gauges and the three reanalysis datasets.The annual precipitation was in continuous fluctuation from 2000 to 2014, and the overall trend of reanalysis datasets was consistent with the precipitation from rain gauges. However, some deviations were found for certain years: the observed rainfall reached a maximum and minimum value in 2011 and 2001, respectively, while ERA-Interim, JRA-55, and NCEP/NCAR-1 were not in agreement: ERA-Interim and JRA-55 peaked in 2003, which meant the rainfall simulation ability should be further enhanced. Meanwhile, changes in the performance of the reanalysis datasets may be related to improvements in algorithms and additional data in recent years.A quantitative evaluation on an annual scale in the QDM is based on the overall performance of the three reanalysis precipitation datasets in Table4.Table 4
CC, BIAS, RMSE, and MAE between three reanalysis precipitation datasets and gauged rainfall at annual scale over the Qinling-Daba Mountains, China (all products passed the significance test at the 99% confidence level).
Index
ERA-Interim
JRA-55
NCEP/NCAR-1
CC
0.54
0.56
−0.19
BIAS
21.78
16.57
−6.31
RMSE
298.21
265.40
437.87
MAE
243.93
214.38
340.83From the evaluation indices, JRA-55 had higher accuracy than ERA-Interim at annual scale with slight advantages. NCEP/NCAR-1 had best BIAS value and worst CC, RMSE, and MAE. Considering the possible mutual cancellation in BIAS, NCEP/NCAR-1 was also regard as the worst product all in all.The spatial distribution of CC for annual precipitation at each gauge over the QDM (Figure7) suggests that ERA-Interim and JRA-55 had CCs beyond 0.5 as a whole, while NCEP/NCAR-1 showed the poorest CCs. Only three stations had CCs for NCEP/NCAR-1 bigger than 0.5, indicating that precipitation in this region cannot be represented on an annual scale. In addition, no obvious pattern was observed for correlation coefficients in the spatial distribution of NCEP/NCAR-1 (Figure 7(c)).Figure 7
Correlation coefficient at each gauge over the Qinling-Daba Mountains for annual precipitation between (a) ERA-Interim and gauges, (b) JRA-55 and gauges, and (c) NCEP/NCAR-1 and gauges.
(a)
(b)
(c)Comprehensively, the values of RMSE and MAE varied with the precipitation accumulation on monthly, seasonal, and annual scales. ERA-Interim and JRA-55 performed better on a monthly scale than an annual scale. On a monthly scale, ERA-Interim had a higher CC and lower MAE values than JRA-55, while JRA-55 had a lower BIAS and RMSE than ERA-Interim. The performance of ERA-Interim and JRA-55 is equally matched. However, JRA-55 was better than ERA-Interim for all indices on an annual scale. Further, ERA-Interim and JRA-55 exhibit a better ability to simulate precipitation in the spring, summer, and autumn than in winter.


### 4.4. Performance in the Wet and Dry Years

To further explore the performance of reanalysis precipitation datasets, based on whether the gauges average annual precipitation in the year is greater or less than the average during 2000 to 2014, the 15-year period was further divided into two groups: wet years and dry years. The wet years include 2000, 2003, 2005, 2009, 2010, 2011, and 2014; the other years between 2001 and 2014 belong to the dry years over the QDM. The PDs and PPDs during the wet and dry years are shown in Tables5 and 6, respectively.Table 5
The PDs and PPDs between ERA-Interim, JRA-55, and NCEP/NCAR-1 and observed precipitation for the wet years.
Reanalysis
ERA-Interim
JRA-55
NCEP/NCAR-1
Absolute precipitation difference (mm)
2000
405.25
259.75
94.72
2003
371.56
192.93
−55.51
2005
125.74
60.4
−48.49
2009
103.6
69.75
−121.62
2010
178.22
97.56
−43.38
2011
39.93
68.77
−242.83
2014
52.86
83.53
−84.66
Percentage of PD (%)
2000
46.18
29.6
10.79
2003
37.61
19.53
−5.62
2005
13.76
6.61
−5.31
2009
11.94
8.04
−14.01
2010
19.32
10.58
−4.7
2011
3.99
6.88
−24.28
2014
6.29
9.93
−10.07Table 6
The PDs and PPDs between ERA-Interim, JRA-55, and NCEP/NCAR-1 and observed precipitation for the dry years.
Reanalysis
ERA-Interim
JRA-55
NCEP/NCAR-1
The absolute precipitation difference (mm)
2001
234.99
233.97
−170.66
2002
269.71
215.85
40.56
2004
283.36
135.59
39.84
2006
196.41
143.75
85.85
2007
320.53
183.57
92.53
2008
167.6
118.2
−9.86
2012
187.36
159.27
−72.32
2013
148.97
119.31
36.18
Percentage of PD (%)
2001
35.4
35.25
−25.71
2002
37.42
29.95
5.63
2004
36.4
17.42
5.12
2006
27.42
20.07
11.98
2007
39.62
22.69
11.44
2008
21.2
14.95
−1.25
2012
24.89
21.16
−9.61
2013
20.03
16.05
4.87The PDs and PPDs for reanalysis precipitation datasets varied in the wet (Table5) and dry years (Table 6). Overestimation was common during the wet and dry years for ERA-Interim and JRA-55. However, NCEP/NCAR-1 has smaller PDs and PDDs both in wet and dry years, especially in wet years, indicating that it has more underestimated years during dry years than wet years.


### 4.5. Precipitation Centroid

The precipitation centroids of rain gauges and ERA-Interim, JRA-55, and NCEP/NCAR-1 reanalysis products were all located in the central region of the QDM and presented an east-west spatial distribution pattern (Figure8). Visually, ERA-Interim (Figure 8(b)) displayed the best agreement with rain gauges due to a close spatial distribution pattern of centroid movement. Moreover, most precipitation centroids of JRA-55 (Figure 8(c)) and NCEP/NCAR-1 (Figure 8(d)) were generally located in the western parts compared with those of rain gauges, which indicated the largest discrepancy among them. According to the definition of precipitation centroid, the centroid will be closer to places with more abundant precipitation, which indicates that JRA-55 and NCEP/NCAR-1 may overestimate precipitation in western parts of QDM. It also can be found from the spatial distribution of BIAS on a monthly scale (Figure 5). The number of stations for large overestimations by JRA-55 and NCEP/NCAR-1 is more than those of ERA-Interim in the western parts of QDM.Figure 8
The precipitation centroids over the Qinling-Daba Mountains during 2000 to 2014 of (a) rain gauge, (b) ERA-Interim, (c) JRA-55, and (d) NCEP/NCAR-1, and (e) the black box represents the data range in a–d.
(a)
(b)
(c)
(d)
(e)The centroid movement distance is accumulated to estimate the magnitude of correspondence between reanalyzed and gauged data as an evaluation indicator (Table7). The total migration distances of gauge, ERA-Interim, JRA-55, and NCEP/NCAR-1 were determined to be 188.8, 177.9, 159.2, and 296.1 km, respectively. The migration distance of JRA-55 was closest to the gauge precipitation, followed by ERA-Interim, while NCEP/NCAR-1 had the largest deviations from the gauges.Table 7
The migration distance (km) of precipitation centroid of gauge and precipitation products.
Year
Gauge
ERA-Interim
JRA-55
NCEP/NCAR-1
2000
2001
19.6
40.3
7.2
55.7
2002
19.6
19.1
8.6
30.3
2003
8.9
12.0
11.1
22.5
2004
7.6
6.8
13.0
2.8
2005
11.4
12.2
10.7
12.3
2006
12.2
2.4
6.2
3.6
2007
8.9
8.0
15.2
9.6
2008
4.9
5.7
5.2
9.2
2009
12.6
3.8
7.4
25.7
2010
8.1
4.5
10.4
28.6
2011
17.7
14.2
21.2
33.8
2012
16.4
3.5
14.0
7.1
2013
17.5
20.6
5.0
26.5
2014
39.2
24.8
24.1
28.4
Sum
204.6
177.9
159.2
296.1


## 4.1. Performance on Monthly Scale

Three reanalysis precipitation datasets were first validated on a monthly scale. To eliminate the influence of the seasonal cycle on CC, the CC of each precipitation dataset was calculated per month (Figure3). All datasets passed the significance test at the 99% confidence level except NCEP/NCAR-1 in certain months. It is clear that ERA-Interim outperformed the other reanalysis precipitation products in most months and had an average CC of 0.64. JRA-55 performed second-best with a mean CC of 0.58. Meanwhile, JRA-55 was better than ERA-Interim in June, July, August, and September, and ERA-Interim had higher agreement with observed precipitation than JRA-55 in the remaining eight months. In addition, JRA-55 simulated precipitation better in months with abundant rainfall. NCEP/NCAR-1 performed worst with the lowest CC (about 0.22) among the evaluated precipitation datasets.Figure 3
The line chart of correlation coefficient (CC) between monthly observed precipitation and ERA-Interim, JRA-55, and NCEP/NCAR-1 precipitation during 2000 to 2014 over the Qinling-Daba Mountains, China.Therefore, the average CCs of the twelve months of a specific year were treated as the overall performance of every single precipitation product on a monthly scale, which are shown in Table2 with the other evaluation indices.Table 2
The average of CC, BIAS, RMSE, and MAE between three reanalysis precipitation datasets and gauged precipitation data on a monthly scale during 2000 to 2014 over the Qinling-Daba Mountains, China.
Index
ERA-Interim
JRA-55
NCEP/NCAR-1
CC
0.64
0.58
0.22
BIAS (%)
21.78
16.57
−6.31
RMSE (mm)
48.56
43.05
64.93
MAE (mm)
30.42
30.81
39.18Overall, ERA-Interim and JRA-55 revealed a similar ability to simulate rainfall for evaluation indices. ERA-Interim had better CC and MAE, and JRA-55 had a better BIAS and RMSE. NCEP/NCAR-1 had the lowest CC and largest RMSE and MAE, suggesting that NCEP/NCAR-1 is the poor performing dataset, even though it had a low BIAS.The spatial distribution of evaluation indices at individual gauges was obtained to investigate the performance of the three precipitation products, as illustrated in Figures4 and 5.Figure 4
Correlation coefficient at each precipitation gauge stations over the Qinling-Daba Mountains for monthly precipitation between (a) ERA-Interim and gauges, (b) JRA-55 and gauges, and (c) NCEP/NCAR-1 and gauges.
(a)
(b)
(c)Figure 5
BIAS (%) at each gauge over the Qinling-Daba Mountains for monthly precipitation between (a) ERA-Interim and gauges, (b) JRA-55 and gauges, and (c) NCEP/NCAR-1 and gauges.
(a)
(b)
(c)The spatial distribution of correlation coefficients suggests that CCs at most sites were greater than 0.5 for ERA-Interim and JRA-55 (Figure4). ERA-Interim and JRA-55 have the same number of stations with CC values higher than 0.7. NCEP/NCAR-1 (Figure 4(c)) showed the worst performance with lower CCs at most stations, where five stations even had CC values lower than 0.5.Furthermore, stations with relatively high CCs were concentrated in the northeastern region of the QDM, which due to the relatively low altitude in the eastern region. The influence of terrain in the eastern region is less than the western and northwestern part of QDM. It is well established that the topographic and orographic influences on precipitation formation and propagation. It is expected that there is less precipitation on the leeward side of the mountain on the western side of the QDM because of the dry-adiabatic decent of air, which leads to lower CCs. It is worth noting that ERA-Interim and JRA-55 had the lowest CC at Wudu station at the same time.ERA-Interim and JRA-55 had similar distributions of BIAS, which are both overestimated the precipitation at the most stations. Comparatively, NCEP/NCAR-1 underestimated rainfall. ERA-Interim, JRA-55, and NCEP/NCAR-1 all had the largest errors at Wudu station with positive BIAS values of 130.4%, 123.3%, and 223.4%, respectively, which may be attributed to the complex terrain in Bailongjiang Valley. Another interesting phenomenon was the underestimation of precipitation at Huashan station for ERA-Interim, JRA-55, and NCEP/NCAR-1. Considering that the elevation of Huashan station is 2054 m a.s.l., the large wind at such a high elevation leads to lower precipitation gauge capture rate [57], and precipitation of reanalysis products is expected to be overestimated. However, it is difficult to determine the contribution of the underestimation of observed precipitation and the overestimation of the reanalysis products, respectively.


## 4.2. Performance on Seasonal Scale

In this study, precipitation was greater during summer (June–August) and autumn (September–November) than in spring (March–May) and winter (December–next February) (Figure2(b)). To understand the seasonal pattern of errors for reanalysis precipitation datasets comprehensively, the overall performance of ERA-Interim, JRA-55, and NCEP/NCAR-1 were evaluated and are shown in Table 3.Table 3
CC, BIAS, RMSE, and MAE between three reanalysis precipitation datasets and gauged precipitation data at the seasonal scale over the Qinling-Daba Mountains, China (all products passed the significance test at the 99% confidence level).
Season
Dataset
CC
BIAS (%)
RMSE (mm)
MAE (mm)
Spring
ERA-Interim
0.67
23.61
69.45
55.79
JRA-55
0.51
35.61
89.65
71.88
NCEP/NCAR-1
0.07
−5.55
120.12
88.11
Summer
ERA-Interim
0.57
27.80
187.92
154.01
JRA-55
0.66
1.86
120.62
95.39
NCEP/NCAR-1
0.12
1.99
211.08
170.28
Autumn
ERA-Interim
0.66
10.38
87.10
67.36
JRA-55
0.70
8.49
83.21
62.20
NCEP/NCAR-1
0.05
−14.92
171.11
128.61
Winter
ERA-Interim
0.49
109.77
36.03
31.26
JRA-55
0.17
209.67
65.95
58.16
NCEP/NCAR-1
0.05
16.03
29.43
21.06All reanalysis datasets displayed higher CC values in spring, summer, and autumn than winter and had lower BIAS values in summer and autumn than spring and winter. Thus, precipitation datasets performed better in warmer and wetter seasons (summer and autumn), which may be conducive to monitoring and predicting geologic hazards caused by heavy rain in a short time period in the QDM. The larger errors in RMSE and MAE in summer and autumn may be due to the fact that rainfall concentrated during those seasons over the QDM (Figure2).It is worth mentioning that JRA-55 coincided worse performing with observed rainfall in spring and winter but performed better in summer and autumn than ERA-Interim, which indicates that it may be better to use JRA-55 for simulating abundant precipitation than ERA-Interim over the QDM. The CCs of NCEP/NCAR-1 were too low to simulate true rainfall, making NCEP/NCAR-1 the worst performance dataset of the three.In summary, all evaluated datasets displayed higher accuracy in summer and autumn than spring and, especially winter, when the performance of the datasets was much worse than the other seasons. ERA-Interim and JRA-55 had similar performance and good agreement with observed precipitation, while NCEP/NCAR-1 showing the poorest performance.


## 4.3. Performance on Annual Scale

The average annual precipitation of each dataset was calculated and compared with the in situ observed precipitation on an annual scale (Figure6).Figure 6
The line chart of regional average annual precipitation from rain gauges and the three reanalysis datasets.The annual precipitation was in continuous fluctuation from 2000 to 2014, and the overall trend of reanalysis datasets was consistent with the precipitation from rain gauges. However, some deviations were found for certain years: the observed rainfall reached a maximum and minimum value in 2011 and 2001, respectively, while ERA-Interim, JRA-55, and NCEP/NCAR-1 were not in agreement: ERA-Interim and JRA-55 peaked in 2003, which meant the rainfall simulation ability should be further enhanced. Meanwhile, changes in the performance of the reanalysis datasets may be related to improvements in algorithms and additional data in recent years.A quantitative evaluation on an annual scale in the QDM is based on the overall performance of the three reanalysis precipitation datasets in Table4.Table 4
CC, BIAS, RMSE, and MAE between three reanalysis precipitation datasets and gauged rainfall at annual scale over the Qinling-Daba Mountains, China (all products passed the significance test at the 99% confidence level).
Index
ERA-Interim
JRA-55
NCEP/NCAR-1
CC
0.54
0.56
−0.19
BIAS
21.78
16.57
−6.31
RMSE
298.21
265.40
437.87
MAE
243.93
214.38
340.83From the evaluation indices, JRA-55 had higher accuracy than ERA-Interim at annual scale with slight advantages. NCEP/NCAR-1 had best BIAS value and worst CC, RMSE, and MAE. Considering the possible mutual cancellation in BIAS, NCEP/NCAR-1 was also regard as the worst product all in all.The spatial distribution of CC for annual precipitation at each gauge over the QDM (Figure7) suggests that ERA-Interim and JRA-55 had CCs beyond 0.5 as a whole, while NCEP/NCAR-1 showed the poorest CCs. Only three stations had CCs for NCEP/NCAR-1 bigger than 0.5, indicating that precipitation in this region cannot be represented on an annual scale. In addition, no obvious pattern was observed for correlation coefficients in the spatial distribution of NCEP/NCAR-1 (Figure 7(c)).Figure 7
Correlation coefficient at each gauge over the Qinling-Daba Mountains for annual precipitation between (a) ERA-Interim and gauges, (b) JRA-55 and gauges, and (c) NCEP/NCAR-1 and gauges.
(a)
(b)
(c)Comprehensively, the values of RMSE and MAE varied with the precipitation accumulation on monthly, seasonal, and annual scales. ERA-Interim and JRA-55 performed better on a monthly scale than an annual scale. On a monthly scale, ERA-Interim had a higher CC and lower MAE values than JRA-55, while JRA-55 had a lower BIAS and RMSE than ERA-Interim. The performance of ERA-Interim and JRA-55 is equally matched. However, JRA-55 was better than ERA-Interim for all indices on an annual scale. Further, ERA-Interim and JRA-55 exhibit a better ability to simulate precipitation in the spring, summer, and autumn than in winter.


## 4.4. Performance in the Wet and Dry Years

To further explore the performance of reanalysis precipitation datasets, based on whether the gauges average annual precipitation in the year is greater or less than the average during 2000 to 2014, the 15-year period was further divided into two groups: wet years and dry years. The wet years include 2000, 2003, 2005, 2009, 2010, 2011, and 2014; the other years between 2001 and 2014 belong to the dry years over the QDM. The PDs and PPDs during the wet and dry years are shown in Tables5 and 6, respectively.Table 5
The PDs and PPDs between ERA-Interim, JRA-55, and NCEP/NCAR-1 and observed precipitation for the wet years.
Reanalysis
ERA-Interim
JRA-55
NCEP/NCAR-1
Absolute precipitation difference (mm)
2000
405.25
259.75
94.72
2003
371.56
192.93
−55.51
2005
125.74
60.4
−48.49
2009
103.6
69.75
−121.62
2010
178.22
97.56
−43.38
2011
39.93
68.77
−242.83
2014
52.86
83.53
−84.66
Percentage of PD (%)
2000
46.18
29.6
10.79
2003
37.61
19.53
−5.62
2005
13.76
6.61
−5.31
2009
11.94
8.04
−14.01
2010
19.32
10.58
−4.7
2011
3.99
6.88
−24.28
2014
6.29
9.93
−10.07Table 6
The PDs and PPDs between ERA-Interim, JRA-55, and NCEP/NCAR-1 and observed precipitation for the dry years.
Reanalysis
ERA-Interim
JRA-55
NCEP/NCAR-1
The absolute precipitation difference (mm)
2001
234.99
233.97
−170.66
2002
269.71
215.85
40.56
2004
283.36
135.59
39.84
2006
196.41
143.75
85.85
2007
320.53
183.57
92.53
2008
167.6
118.2
−9.86
2012
187.36
159.27
−72.32
2013
148.97
119.31
36.18
Percentage of PD (%)
2001
35.4
35.25
−25.71
2002
37.42
29.95
5.63
2004
36.4
17.42
5.12
2006
27.42
20.07
11.98
2007
39.62
22.69
11.44
2008
21.2
14.95
−1.25
2012
24.89
21.16
−9.61
2013
20.03
16.05
4.87The PDs and PPDs for reanalysis precipitation datasets varied in the wet (Table5) and dry years (Table 6). Overestimation was common during the wet and dry years for ERA-Interim and JRA-55. However, NCEP/NCAR-1 has smaller PDs and PDDs both in wet and dry years, especially in wet years, indicating that it has more underestimated years during dry years than wet years.


## 4.5. Precipitation Centroid

The precipitation centroids of rain gauges and ERA-Interim, JRA-55, and NCEP/NCAR-1 reanalysis products were all located in the central region of the QDM and presented an east-west spatial distribution pattern (Figure8). Visually, ERA-Interim (Figure 8(b)) displayed the best agreement with rain gauges due to a close spatial distribution pattern of centroid movement. Moreover, most precipitation centroids of JRA-55 (Figure 8(c)) and NCEP/NCAR-1 (Figure 8(d)) were generally located in the western parts compared with those of rain gauges, which indicated the largest discrepancy among them. According to the definition of precipitation centroid, the centroid will be closer to places with more abundant precipitation, which indicates that JRA-55 and NCEP/NCAR-1 may overestimate precipitation in western parts of QDM. It also can be found from the spatial distribution of BIAS on a monthly scale (Figure 5). The number of stations for large overestimations by JRA-55 and NCEP/NCAR-1 is more than those of ERA-Interim in the western parts of QDM.Figure 8
The precipitation centroids over the Qinling-Daba Mountains during 2000 to 2014 of (a) rain gauge, (b) ERA-Interim, (c) JRA-55, and (d) NCEP/NCAR-1, and (e) the black box represents the data range in a–d.
(a)
(b)
(c)
(d)
(e)The centroid movement distance is accumulated to estimate the magnitude of correspondence between reanalyzed and gauged data as an evaluation indicator (Table7). The total migration distances of gauge, ERA-Interim, JRA-55, and NCEP/NCAR-1 were determined to be 188.8, 177.9, 159.2, and 296.1 km, respectively. The migration distance of JRA-55 was closest to the gauge precipitation, followed by ERA-Interim, while NCEP/NCAR-1 had the largest deviations from the gauges.Table 7
The migration distance (km) of precipitation centroid of gauge and precipitation products.
Year
Gauge
ERA-Interim
JRA-55
NCEP/NCAR-1
2000
2001
19.6
40.3
7.2
55.7
2002
19.6
19.1
8.6
30.3
2003
8.9
12.0
11.1
22.5
2004
7.6
6.8
13.0
2.8
2005
11.4
12.2
10.7
12.3
2006
12.2
2.4
6.2
3.6
2007
8.9
8.0
15.2
9.6
2008
4.9
5.7
5.2
9.2
2009
12.6
3.8
7.4
25.7
2010
8.1
4.5
10.4
28.6
2011
17.7
14.2
21.2
33.8
2012
16.4
3.5
14.0
7.1
2013
17.5
20.6
5.0
26.5
2014
39.2
24.8
24.1
28.4
Sum
204.6
177.9
159.2
296.1


## 5. Discussion

### 5.1. Poor Performance in Winter

In terms of time, all three reanalysis precipitation datasets showed poor performance in winter on a seasonal and monthly scale. Standard Chinese rain gauges lack windproof and automatic heating devices; the solid precipitation measured is artificially melted into water immediately after the snow events. Considering the complexity of mountainous areas such as QDM, the interference by wind may cause only half of the actual precipitation to be represented by observed solid precipitation in rain gauges without a windproof device [57]. Moreover, the wetting losses caused by snow probably have an impact on the observation results in winter [58]. Large underestimations probably exist in observed winter precipitation and most likely prevent reanalysis datasets from agreeing well with the observation data. The performance of different precipitation products needs to further evaluate by measuring more winter precipitation by more windproofed weighted rain gauges.


### 5.2. Poor Performance at Wudu Station and Huashan Station

The three datasets evaluated in this work display bigger errors and larger deviations at Wudu station than the other stations, for which the complex terrain may be responsible. Wudu station is located in the Bailongjiang River valley with an altitude of 1079 m a.s.l., while the mountains on both sides are all above this elevation, and the peak of the mountains are above 2500 m a.s.l. Although the concave terrain allows the rain gauge at Wudu station to obtain accurate rainfall in the valley, it cannot reflect the true precipitation information in the surrounding areas to some extent. Some precipitation recorded at Wudu station was probably caused by local convection, but not over a large range. On the other hand, the difference between the actual altitude and the altitude in different reanalysis data probably contributes to the relatively large errors at Wudu station, where the complex terrain cannot be represented by coarse resolution reanalysis data.The elevation of Huashan station is 2064 m a.s.l., which almost reaches the elevation of peak of Huashan Mountain (2154 m a.s.l.). The three reanalysis precipitation datasets underestimated the precipitation at Huashan station at the same time, which also can be explained by the large difference between the actual altitude and the altitude in different reanalysis data. This also indicates that there are precipitation gradients around Huashan stations, although they are difficult to be detected in current in situ observation and satellite precipitation products. More intense in situ precipitation observation network will help to obtain the local precipitation gradients.


### 5.3. Reasons for Different Performance of Three Products

It needs to note that various reanalysis datasets may have varied performance on different scales and for each evaluation index. Harada et al. [38] found that JRA-55 had better correlation coefficients than NCEP/NCAR-1 in Eurasia and North America, which is in line with our research results. Wang and Zeng [59] found GLDAS has the best overall performance for daily and monthly precipitation, while ERA-40 and MERRA have the highest CCs. In this study, although the BIAS values of NCEP/NCAR-1 were good, the other indices of NCEP/NCAR-1 are relatively poor. The reasons include the following:ERA-Interim and JRA-55 merged precipitation data from observed stations, while no rain observations were included for assimilation of the NCEP/NCAR-1 model. However, how many and which stations in the QDM were involved in ERA-Interim and JRA-55 are difficult to obtain and cannot be excluded in the evaluation. It seems unfair to compare ERA-Interim, JRA-55, and NCEP/NCAR-1 by the same observation data. On the other hand, the coarsest spatial resolution may be another reason for the poor performance of NCEP/NCAR-1. Meanwhile, the advantages of the four-dimensional variational analysis over three-dimensional variational analysis model is potentially one of reasons why ERA-Interim and JRA-55 showed higher agreement than NCEP/NCAR-1 [42], which has also been proven in previous studies [60, 61]. Moreover, due to the differences in the original resolution of different reanalysis data, the interpolation method likely has some impact on the evaluation.It should also be noted that the observed precipitation also probably has some uncertainty. The general problem of representativeness is particularly acute in the measurement of precipitation, and precipitation measurements are particularly sensitive to exposure, wind, and topography. Although both the meteorological stations are observed with the manually standard process in the standard field with grass land cover, the local factors such as terrain and wind are still different to considered and corrected. Many different studies [62–64], among many, have concluded that automated gauges have many errors associated with them, among which some errors as high as 80% under extremely detrimental conditions. There is a whole entire field of quality controlling gauge data to further quality control radar-derived rain rates. Therefore, these flaws should be fixed in future studies on the evaluation of multiple precipitation datasets, including use of potentially more effective hydrological models.


### 5.4. Comparisons with Previous Study in QDM

Based on this study and Wang’s previous study [27] on satellite-based precipitation datasets in QDM, ITPCAS has the highest accuracy among satellite precipitation datasets, such as CMORPH, GPCP-2, and GPCC compared against the rain-gauge observations. It also has the better accuracy with higher CC, lower BIAS, RMSE, and MAE than the three reanalysis precipitation products in this study. The main reason is probably because ITPCAS integrate more rain gauge observation data, which also used for the baseline of the evaluation. Thus, it was excluded in the evaluation. This study indicates that high precision precipitation datasets, including ERA-Interim and JRA-55 in this study, can be applied into hydrological models, and the applicability and adaptability of different precipitation datasets provide an important basis for hydrometeorological simulation and other applications in the QDM, which will also help to understand the water cycle in the whole Tibetan Plateau.


## 5.1. Poor Performance in Winter

In terms of time, all three reanalysis precipitation datasets showed poor performance in winter on a seasonal and monthly scale. Standard Chinese rain gauges lack windproof and automatic heating devices; the solid precipitation measured is artificially melted into water immediately after the snow events. Considering the complexity of mountainous areas such as QDM, the interference by wind may cause only half of the actual precipitation to be represented by observed solid precipitation in rain gauges without a windproof device [57]. Moreover, the wetting losses caused by snow probably have an impact on the observation results in winter [58]. Large underestimations probably exist in observed winter precipitation and most likely prevent reanalysis datasets from agreeing well with the observation data. The performance of different precipitation products needs to further evaluate by measuring more winter precipitation by more windproofed weighted rain gauges.


## 5.2. Poor Performance at Wudu Station and Huashan Station

The three datasets evaluated in this work display bigger errors and larger deviations at Wudu station than the other stations, for which the complex terrain may be responsible. Wudu station is located in the Bailongjiang River valley with an altitude of 1079 m a.s.l., while the mountains on both sides are all above this elevation, and the peak of the mountains are above 2500 m a.s.l. Although the concave terrain allows the rain gauge at Wudu station to obtain accurate rainfall in the valley, it cannot reflect the true precipitation information in the surrounding areas to some extent. Some precipitation recorded at Wudu station was probably caused by local convection, but not over a large range. On the other hand, the difference between the actual altitude and the altitude in different reanalysis data probably contributes to the relatively large errors at Wudu station, where the complex terrain cannot be represented by coarse resolution reanalysis data.The elevation of Huashan station is 2064 m a.s.l., which almost reaches the elevation of peak of Huashan Mountain (2154 m a.s.l.). The three reanalysis precipitation datasets underestimated the precipitation at Huashan station at the same time, which also can be explained by the large difference between the actual altitude and the altitude in different reanalysis data. This also indicates that there are precipitation gradients around Huashan stations, although they are difficult to be detected in current in situ observation and satellite precipitation products. More intense in situ precipitation observation network will help to obtain the local precipitation gradients.


## 5.3. Reasons for Different Performance of Three Products

It needs to note that various reanalysis datasets may have varied performance on different scales and for each evaluation index. Harada et al. [38] found that JRA-55 had better correlation coefficients than NCEP/NCAR-1 in Eurasia and North America, which is in line with our research results. Wang and Zeng [59] found GLDAS has the best overall performance for daily and monthly precipitation, while ERA-40 and MERRA have the highest CCs. In this study, although the BIAS values of NCEP/NCAR-1 were good, the other indices of NCEP/NCAR-1 are relatively poor. The reasons include the following:ERA-Interim and JRA-55 merged precipitation data from observed stations, while no rain observations were included for assimilation of the NCEP/NCAR-1 model. However, how many and which stations in the QDM were involved in ERA-Interim and JRA-55 are difficult to obtain and cannot be excluded in the evaluation. It seems unfair to compare ERA-Interim, JRA-55, and NCEP/NCAR-1 by the same observation data. On the other hand, the coarsest spatial resolution may be another reason for the poor performance of NCEP/NCAR-1. Meanwhile, the advantages of the four-dimensional variational analysis over three-dimensional variational analysis model is potentially one of reasons why ERA-Interim and JRA-55 showed higher agreement than NCEP/NCAR-1 [42], which has also been proven in previous studies [60, 61]. Moreover, due to the differences in the original resolution of different reanalysis data, the interpolation method likely has some impact on the evaluation.It should also be noted that the observed precipitation also probably has some uncertainty. The general problem of representativeness is particularly acute in the measurement of precipitation, and precipitation measurements are particularly sensitive to exposure, wind, and topography. Although both the meteorological stations are observed with the manually standard process in the standard field with grass land cover, the local factors such as terrain and wind are still different to considered and corrected. Many different studies [62–64], among many, have concluded that automated gauges have many errors associated with them, among which some errors as high as 80% under extremely detrimental conditions. There is a whole entire field of quality controlling gauge data to further quality control radar-derived rain rates. Therefore, these flaws should be fixed in future studies on the evaluation of multiple precipitation datasets, including use of potentially more effective hydrological models.


## 5.4. Comparisons with Previous Study in QDM

Based on this study and Wang’s previous study [27] on satellite-based precipitation datasets in QDM, ITPCAS has the highest accuracy among satellite precipitation datasets, such as CMORPH, GPCP-2, and GPCC compared against the rain-gauge observations. It also has the better accuracy with higher CC, lower BIAS, RMSE, and MAE than the three reanalysis precipitation products in this study. The main reason is probably because ITPCAS integrate more rain gauge observation data, which also used for the baseline of the evaluation. Thus, it was excluded in the evaluation. This study indicates that high precision precipitation datasets, including ERA-Interim and JRA-55 in this study, can be applied into hydrological models, and the applicability and adaptability of different precipitation datasets provide an important basis for hydrometeorological simulation and other applications in the QDM, which will also help to understand the water cycle in the whole Tibetan Plateau.


## 6. Conclusions

QDM is a unique region where precipitation can significantly be impacted by the terrain which has serious impacts to the local community by environmental, ecological, and biological processes. Three reanalysis precipitation datasets, including ERA-Interim, JRA-55, and NCEP/NCAR-1, were evaluated over the QDM against rain gauge data from 2000 to 2014 on monthly, seasonal, and annual scales. Based on all results, some conclusions can be made:Overall, the performance of ERA-Interim is close to that of JRA-55 with higher CC above 0.5 and lower RMSE less than 50 mm in monthly scale, while NCEP/NCAR-1 has the worst performance on a monthly scale and annual scale. However, the NCEP/NCAR-1 has the least BIAS with the observed precipitation in an annual scale in QDM.All reanalysis datasets performed better in spring, summer, and autumn than in winter. JRA-55 had a better agreement with rain gauge data in summer and autumn, while ERA-Interim exhibited a higher agreement in spring and winter in QDM.The advantages of involving more precipitation observation stations are probably the main reason of the different performance of three precipitation reanalysis products, and the benefit of a four-dimensional variational analysis model over a three-dimensional variational analysis model may be another reason.The evaluation on different precipitation products is very important to understanding the spatial-temporal distribution of precipitation in QDM, which is critical to simulation the hydrological processes and water resource management in QDM, where is the main water source of Xian city. Enhancing the precipitation measuring accuracy, especially in winter, and increasing the measuring stations are still needed to further evaluation the different precipitation products in QDM.



---

*Source: 7698171-2019-01-23.xml*