dataesr/scientific-paragraphs-categorization huggingface dataset:
In addition, this paper presents the validation of the methodology over a large set of catchments (41 river basins), covering a large area of South America. This paper addresses the following questions: i) How can we use hydrological data and hydrological modeling to blend different observed precipitation datasets and validate the combined product in order to have better precipitation data for hydrological simulation and flow forecasting? ii) How can we extend the combined precipitation product over a period other than the common period of the different observed precipitation time series to foster its use in (near)real-time applications? This paper illustrates how we can use hydrological data and modeling as a tool to quantify the uncertainty of precipitation estimates, combine different datasets and validate the combined product with the final goal of obtaining a precipitation product relevant to (near)real-time hydrological applications.
Our methodology is applied to a large set of 41 river basins that are relevant to the hydropower sector in Brazil and neighboring countries. The TRMM-MERGE and CPC precipitation datasets are used to generate a combined precipitation product. The dataset obtained is compared with the benchmark MSWEP and evaluated in terms of how hydrological simulations compare with observed discharges. In Section 2, the methodology and data used are presented. In Section 3, the results obtained are provided. Section 4 is a discussion of the results, and the conclusions and planned future studies are presented in Section 5.
Study area and data The study area comprises 41 river basins distributed in different climatic regions within Brazil and neighboring countries (Fig. 1 ). The river basins vary in size, with areas ranging from 9300 to 382,000 km 2. The study area extends from the north (Jari River, Madeira River, Xingu River, Tapajos River, Tocantins River, and others) to the south of Brazil (Iguaçu River, Uruguay River, and others) and includes river basins located in the central part of the country (Paraná River, Grande River, São Francisco River, etc. ). The 41 river basins provide inflow to 30 hydropower plants (HPPs). For each river basin, daily areal precipitation was obtained from the TRMM-MERGE, the CPC-NOAA, and the MSWEP datasets (Table  1 ). Daily areal precipitation time series were calculated using the shape file of the basins and averaging all grid data points falling inside each river basin considered.
Daily discharge data were obtained from the ONS (National Operator of the Electric System; downloaded from https://sintegre. ons. org. br). They correspond to the official HPP natural flow time series and are compiled annually by the national operator. In some cases, it is the ONS that takes into account the regularization effects of the reservoirs and adds their evaporation as well as any water uses to obtain the natural flows of the reservoir. In these cases, the flows are also validated by the electricity generators involved in the process (for more details, see  ONS, 2005). Data availability depends on each river basin. For this study, the discharge dataset used covers the period 1979-2018 for all studied river basins.
Methodological steps To select a better real-time precipitation dataset among the two available independent data sources and a dataset that is a combination of the two, and to validate the selection and blending procedure using a hydrological model and observed streamflow, an experiment was designed with six basic methodological steps. Step 1: For each river basin, the daily areal precipitation time series (average over the area of the river basin) is estimated for each precipitation data source. The daily observed flow data are extracted for the same period at each basin. In addition, the total annual precipitation, and the annual mean of observed daily flows are calculated to perform the analysis based on the water balance relationship (Step 2). Step 2: The uncertainty of each precipitation data source is identified and quantified by using empirical functions that relate the total annual precipitation amounts and the observed annual mean flows.
The weights used to build the combined daily precipitation dataset are obtained based on the uncertainty quantified from the annual water balance, as detailed in Section 2. 3. Step 3: Since the two independent precipitation data sources cover different time periods, a procedure is used to extend the combined precipitation dataset over the longest period using the double-mass curve, as described in Section 2. 4. Step 4: For each daily precipitation dataset (the two independent sets and the combined one), the hydrological model is calibrated at each river basin by first applying the traditional split-sample test, which divides the time series (October 1998 to September 2017) into two periods for calibration and validation. This allows to assess the robustness of the model when simulating streamflows over a period independent of the calibration period.
We also apply the calibration procedure over the entire data period to evaluate how the parameters of the model change with the length of the time period used for calibration. This step is detailed in Section 2. 5. Step 5: For each precipitation dataset and river basin, the daily flows simulated by the hydrological model (with parameters calibrated over the entire data period) are evaluated against the observed flows at the outlet of the basin. We use a variety of performance metrics, as described in Section 2. 6. Step 6: The extended combined precipitation dataset is evaluated against a benchmark of observed precipitation, and simulated discharges using the new dataset are evaluated against observed discharges.
The error ε i is evaluated for each given year using the empirical function that relates each annual precipitation P i to the annual flow Q. The evaluation is based on the standard deviation of the errors of each precipitation source P i , and is computed over the time series of annual errors. The standard deviation estimates obtained were used to weight the proportion of each precipitation data source to create a combined precipitation dataset. The weights were based on the proportion of the standard deviation of the errors of one source with respect to the sum of standard deviations of the errors of both sources. The weights were obtained from the following equations: W j = 1 - σε j σε j + σε k (3) W k = 1 -W j (4) where W j is the weight of the precipitation data source P i=j , W k is the weight of the precipitation data source P i=k , σε j is the standard deviation of the annual errors of P i=j , and σε k is the standard deviation of the annual errors of source P i=k , at each basin.
The main idea behind this methodology is not to identify the precipitation data source that displays the best correlation between P i and Q. The goal is to access the uncertainty of each source to make it possible to give a higher weight to the source with smaller uncertainty when the two sources of precipitation data are combined in one unique dataset. Once the weights of each precipitation data source are obtained, they are applied to the daily precipitation time series to generate the combined daily precipitation dataset at each basin, according to Eq. (  5 ): DP WC = DP j. W j + DP k. W k (5) where DP WC is the daily precipitation of the weighted combination of the two precipitation data sources, DP j and DP k.
Double-mass curve method to extend the combined precipitation dataset Once the combined precipitation method is applied to the two different precipitation data sources, a combined precipitation dataset for the same period of the shorter precipitation data source is obtained. It is thus necessary to extend, in a consistent way, this combined precipitation time series to also cover the period of the longer available data source. In this study, the double-mass curve method developed by the US Geological Survey was used  (Searcy et al. , 1960). The method is commonly used in data analyses to check the consistency of hydrometeorological data and adjust the data for any inconsistencies. It is based on comparing the time series of accumulated values at a single data station (or a given data source) with those given by data from other stations in a surrounding area (or other data sources) during the same period.
The theory of the double-mass curve is based on the fact that a graph of the accumulation of one time series against the accumulation of another time series, during the same period, plots as a straight line as long as the data are proportional; the slope of the line represents the constant of proportionality (b) between them. Applied to this study, it gives Eq. (  6 ): P c = P k. b (6) where P c represents the precipitation variable of the combined precipitation dataset, and P k is the precipitation variable coming from the longer time series of the two sources of precipitation used to produce the combined dataset. A break in the slope of Eq. (  6 ) indicates a change in the constant of proportion and the presence of inconsistencies. In this study, the double-mass curve was applied to annual values of total precipitation as follows.
• For each river basin, for the common period of  TRMM-MERGE and CPC-NOAA data (i. e. , 1998-2017) , the graph of accumulated annual values of the combined dataset is plotted against the graph of the CPC-NOAA dataset, which is the longer precipitation dataset. • The years with a wide discrepancy around the normal tendency are removed, with the objective of having the most representative parameter to extend the time series. • The constant of proportionality obtained with this correlation is used to multiply the past period of the longer data source, obtaining thus a complete time series of daily precipitation back until 1979, in order to represent the extension of the combined precipitation dataset. To compare the combined precipitation dataset with the benchmark precipitation (MSWEP dataset), and to evaluate how the dispersion of the error varies in time, the standard deviation of the annual precipitation errors (observed precipitation minus the value calculated with the empirical functions of Eq.
HEC-HMS hydrological modeling and flow analyses To validate the combined precipitation dataset obtained, a hydrological model was used to evaluate the accuracy of the simulated discharges when this dataset drives the model. The hydrological model used in this study is the one proposed in the suite of the HEC-HMS model  (Feldman, 2000). It is a flexible and user-friendly suite of models that can be used for many hydrological applications, such as urban and rural flooding, flood warning systems, water uses planning, etc. The models can be applied to simulate a single flow event or to perform continuous hydrological simulations in a semi-distributed modeling configuration. The modules can be chosen according to the needs of the user, the application envisaged and the expected accuracy  (Feldman, 2000; Najim, 2013).
The objective function used for the Clark unit hydrograph parameters was the Minimum of Peak-Weighted RMS Error and, for the loss parameters, the Minimum Sum of Squared Residuals  (Diskin and Simon, 1977). Since the objective functions are more sensitive to the volumes and peaks of the hydrographs, it is sometimes necessary to adjust the base flow parameters to have a better fit of the recession of the hydrographs, after running the model optimization. The calibration process is repeated three times for each calibration period (Fig. 2 ), for each precipitation data source and for the combined precipitation dataset, and for each river basin.
Evaluation metrics To analyze the performance of the hydrological models run in this study (HEC-HMS model calibrated with the different observed precipitation datasets), the Nash-Sutcliffe efficiency -NSE  (Nash, 1970) , the Kling-Gupta efficiency -KGE  (Gupta et al. , 2009) , the Mean Absolute Error -MAE, and the R 2 (coefficient of determination) were used as performance indicators. These metrics are used to compare the daily simulated flows with the daily observed flows. They were evaluated for the calibration-independent time periods defined in Fig. 2  (October 1998 to September 2008; October 2008 to September 2017). In our evaluation, the performance indicators obtained for the validation periods 1 and 2 were used to evaluate the performance of the hydrological model at each river basin.
Standard deviation of the annual precipitation errors Fig. 3  shows the boxplot distribution of the standard deviation of the annual precipitation errors (ε i from Eq. (  2 )) obtained from the two sources of precipitation data and the combined precipitation dataset. The standard deviation values were evaluated over all 41 river basins and 19 years of data (October 1998 -September 2017), and the linear function was the best fit in all basins, which makes sense since the annual water balance was represented by a linear function in Eq. (  1 ). The standard deviation of annual precipitation errors from the benchmark MSWEP dataset, for all river basins and the period from 1998 to 2017, is also shown as reference. Table  2  shows the results for each river basin, as well as the weights applied when combining the two precipitation data sources.
The dispersion of the annual precipitation errors is reduced when the different precipitation data sources are combined. The combined precipitation dataset shows a lower standard deviation of the annual precipitation errors than the original datasets on 15 out of 41 river basins, and its boxplot distribution is closer to the reference MSWEP dataset. Although the median value of the error standard deviation is close to the TRMM-MERGE dataset median value, the interquartile range between the 75th and 25th percentiles is smaller in the combined dataset. This measure of dispersion of the precipitation errors around the empirical function gives an idea of the uncertainty of each precipitation dataset. In the combined dataset more weight is given to the precipitation dataset with lower uncertainty.
Hydrological model performance The three precipitation datasets (TRMM-MERGE, CPC and Combined) were used to calibrate and validate the HEC-HMS hydrological model in all 41 river basins. Fig. 4  shows the density functions representing the distribution of the four performance metrics analyzed: KGE, NSE, MAE (calculated in terms of specific discharge, in l/s. km 2 to permit comparison of the values in basins with different sizes), and R 2. The values obtained in calibration (validation) are in red (blue). Table  3  shows the median values of the metrics when considering all river basins, calibration, and validation periods (as defined in Fig. 2 ), and for each precipitation dataset. For most of the basins, the difference in performance metrics between calibration and validation is less than 10%.
The calibrated models can be considered to perform well over the large river basin dataset of this study. For most of the 41 river basins, the NSE and KGE criteria are higher than 0. 60, and the R 2 coefficient is higher than 0. 55. Table  3  shows that median values of NSE and KGE vary from 0. 71 to 0. 81 in calibration, and from 0. 68 to 0. 78 in validation. Median values of R 2 coefficient in calibration and validation are also very close. These results indicate that the model calibration is sufficiently robust to represent the characteristics of the basins in different periods. Fig. 5  also shows that the combined dataset results in a more consistent calibration and validation performance, with sharper curves and fewer outliers of performance. The combined dataset is also the one that presents the highest median values of NSE, KGE and R 2 and the lowest median values of MAE (Table  3 ).
Selection of the best precipitation dataset After the evaluation and validation of the performance of the hydrological model for each river basin, the best performing precipitation dataset was evaluated in terms of simulated discharges. The simulations were made using the model with parameters calibrated with each precipitation dataset and over the entire period (October 1998 -September 2017). Fig. 5  shows the MAE, KGE, NSE, and R 2 performance metrics obtained for the 41 river basins and the three sources of precipitation data (TRMM-MERGE, CPC, and the combined precipitation dataset). In general, the simulated discharges using the CPC precipitation dataset clearly performed worse for the four indicators analyzed. The simulated discharges using the combined precipitation dataset performs best for almost all basins.
When the scores are not the best for this dataset in a river basin, the differences are less than 10 % in the majority of the cases (not shown). The best performance of the combined dataset is clearly shown with the NSE, MAE and R 2 criteria. In terms of the KGE criterion, 50 % of the river basins present values higher than 0. 79 when using the combined and the TRMM-MERGE precipitation datasets. There are small differences in KGE between these two datasets, although the use of the combined dataset resulted in fewer outliers in terms of performance and notably.
Extension of the combined precipitation dataset Once the evaluation showed that the combined precipitation of the two precipitation data sources achieved the best performance  over the common period of data (October 1998 -September 2017), the next step was to use the double-mass procedure (Section 2. 4) to extend the combined dataset. The correlation of the double-mass curve between the longest dataset (CPC) and the combined dataset was used to extend the period of the combined dataset until 1979 (first year of data available in the longest CPC dataset). To validate the extended dataset obtained, the hydrological models were run for each river basin, with parameters calibrated for the period 1998-2017. The performance of the streamflow simulations of this recent period was compared with the performance of the simulations obtained over the extended (validation) period of 1979-1997.
Comparison of the combined precipitation dataset with the benchmark The combined precipitation dataset was evaluated against the benchmark MSWEP dataset. Fig. 7  shows the evolution in time of the normalized standard deviation of the annual precipitation errors obtained for 1980-2017 in a five-year moving window. The thickest lines represent the median of the values for all river basins and the shadowed areas show the 25th and 75th percentiles (variability among river basins). The extended period of the combined precipitation (before 1998) tends to have a behavior closer to the benchmark. It shows similar standard deviation of annual precipitation errors, but higher values: median values are closer to 6% versus median values closer to 5 % for the benchmark. The combined precipitation dataset does not exhibit a visible temporal trend.
However, the standard deviation of the precipitation errors decreases after the year 2000, to increase again just a few years later. The standard deviation of the MWSEP annual precipitation errors is smaller and more stable over time comparatively to the combined precipitation dataset. It also shows less variability among river basins. After around 1998, there is a reduction in the MSWEP error standard deviation, probably due to the use of satellite observations to improve the estimates of precipitation. After 2008, median values tend to increase again to values similar to those before 1998, although the variability among river basins remain reduced. When the entire period  is considered, the combined precipitation dataset has a higher median and variability among river basins than the benchmark, but the difference can be considered small, especially considering that the combined precipitation is a real-time dataset, subject to errors that might be corrected afterwards, and the level of consistency is not comparable to a reanalysis dataset such as MSWEP.
Examples of simulated hydrographs using the combined precipitation dataset As showed in Sections 3. 2 and 3. 3, the combined precipitation dataset displayed good performance for the period used to build it from the two existing data sources, i. e. , 1998-2017, with NSE and KGE values above 0. 75. Fig. 8  shows three examples of the simulated flows using the combined precipitation dataset for the complete period generated from this study, i. e. , from October 1st, 1979 to September 30th, 2021. Three river basins were selected to represent the southern region (HPP Foz do Chapecó at the Uruguai River), the southeast region (HPP Emborcação at the Paranaíba River), and the north region (HPP Santo Antônio at the Madeira River). In the southern region, the weather does not present a clear seasonality and peaks of flow may occur in any month of the year.
The simulated flows tend to underestimate the highest peaks and overestimate the lowest flows (NSE = 0. 78). In the southeast region, seasonality is present, and the wet season is from November to March. The simulated flows tend to overestimate the flow peaks for the period 1983-1993 and underestimate them for 2005-2010. Despite these tendencies, this river basin also presents a good statistical fit, with NSE value of 0. 76. In the north region, the biggest basin of our dataset is shown. A clear seasonality is observed, with the wet season from December to May. The simulated flows display an overall better and more stable performance, with a good fit to flow peaks and flow recession periods (NSE = 0. 87).
Discussion This study proposes a methodology that allows us to combine, extend and validate two precipitation datasets, based on precipitation gauges and satellite-based products (CPC and TRMM-MERGE), by using observed and simulated discharges at river basin outlets. By relying on discharge data and hydrological modeling to process the global gridded precipitation products, it provides a framework that can be applied, in particular, in large areas where ground networks of rain gauges, often considered as the most Fig. 7. Normalized standard deviation of the annual precipitation errors with the combined precipitation dataset (red) and the benchmark MSWEP dataset (blue). Median values (lines) and 25th and 75th percentiles (shadowed areas) are calculated in a five-year moving window and over 41 river basins.
, 2009). The study area also comprises a weather type with no seasonality: the subtropical weather at the south region of the continent, which presents wet and dry periods in any month of the year  (Garreaud et al. , 2009). This large domain of application illustrates the ability of the approach to provide the means to combine precipitation datasets under a variety of hydrological and climatic conditions. The approach proposed in this study also allow us to take advantage of the quality control and knowledge already developed in each precipitation source, such as the five levels quality control at the CPC dataset  (Chen. M, et al. , 2008). Observed discharges are used in the computation of the annual water balance in order to quantify the uncertainties of each precipitation source. This is done by using the standard deviation of the errors, which are then used to define the weights that will generate the blending of the different precipitation datasets at each river basin.