Proc. of Monitoring, Prediction and Mitigation of Water-Related Disasters (MPMD2005), pp. 281-286, 2005

Prediction Uncertainty Identification in Hydrologic Modeling: A Methodology
for Uncertainty Recognition and Quantification
Shen Chiang1, Yasuto Tachikawa2 and Kaoru Takara2
1
2

Dept. of Civil Eng., Kyoto University, Yoshida Hon-machi, Sakyo-ku, Kyoto, 606-8501, Japan
Disaster Prevention Research Institute, Kyoto University, Gokasho, Uji, Kyoto 611-0011, Japan

ABSTRACT: We propose a methodology to identify prediction uncertainty through recognizing andquantifying the different uncertainty sources in a hydrologic model. Statistical second moment is used as ameasure of uncertainty; also an index which originated from Nash coefficient of efficiency named ModelStructure Indicating Index (MSII) is proposed to evaluate the model predicting uncertainty. The results showthat MSII can well reflect the goodness of model structure, while a larger value of MSII indicating a poorer
structure of hydrologic model. The index can be used as a tool for implementing model quantitative
comparison.

1INTRODUCTION

With rapid advances in computing technology, remote sensing, GIS and DBMS, the role of hydrologic models isenhanced as a tool in planning, decision making and tends to incorporating with other process models such likeeconomic, social, political, administrative, and judicial models. Thus, the watershed hydrologic models will
become a component in a larger management strategy. Furthermore, these models will become more global,
not only in the sense of spatial scale but also in the sense of hydrologic details (Singh and Woolhiser, 2002).
Therefore, a methodology to assess the error, uncertainty and adequacy of adopting hydrologic models in
specific purpose is needed. The present study provides a methodology for model comparison and selection
through model uncertainty recognition and quantification.
For many years hydrologists have been interested in the effects of various uncertainties on the accuracy andreliability of the estimation of catchment hydrological variables such as peak flow and flood volume (Høybyeand Rosbjerg, 1999). Among early contributions, some of them focus on the rainfall uncertainty and its
influence to the runoff (e.g., Storm et al., 1989); some of them focus on the sensitivity of model structure due tothe input error (e.g., Singh and Woolhiser, 1976). Recent researches relating to hydrologic model uncertainty
most refer to parameter uncertainty identification, the procedure of parameter calibration, and their impact tosimulation result (Freer et al., 1996; Kuczera and Parent, 1998). Among them, Generalized Likelihood
Uncertainty Estimation (GLUE) Methodology (Freer et al., 1996) offers a path of identifying parameter
uncertainty. Nevertheless, parameter equifinality became the conclusion of GLUE; uncertainty related to input
data and other factors are excluded. Even it is said that they could also be included in GLUE but this has notnormally been done (Beven, 2001).
The performance of hydrologic models is profoundly affected by the sources of uncertainty, briefly they are:observed data, data for model calibration, parameter space, and model structure. Underestimating or
misunderstanding of these uncertainty sources and the interrelation among them may cause tremendous
misleading on interpreting the result of hydrologic models. However, recognizing and quantifying thesedifferent sources of uncertainty in hydrologic models has received little attention in the research literature.
In this study, a methodology is proposed to recognize and quantify the different uncertainty sources.
Firstly, Monte Carlo simulation method is applied to add bias item in model input data series, then rainfallrealizations, parameter space, and model outcomes under different bias level are acquired. Secondly, byexamining the counter relationship between model simulation outcomes, calibration outcomes and observed
watershed response series, an uncertainty structure is recognized. Finally, parameter uncertainty, calibration
uncertainty, and model structure uncertainty caused by input data uncertainty are recognized, separated, andquantified through the methodology.
Statistical second moment is used as a measure of uncertainty, also an index which originated from Nashcontents page

coefficient named Model Structure Indicating Index (MSII) is proposed to quantify model structure uncertainty
which can be used as a tool for implementing model quantitative comparison (selection). The results show thata larger value of MSII indicating a poorer structure of hydrologic model in a dynamic manner, that is,incorporating the MSII to the input uncertainty.
2UNCERTAINTY RECOGNITION IN HYDROLOGICAL MODELINGIn this study, prediction uncertainty which came from the four kinds of sources mentioned previously isclassified into four categories: system uncertainty, entire uncertainty, inherent uncertainty, and structure
uncertainty. The definition and the procedure to recognize them are described below.
2.1 System uncertainty
Even it is well known that a hydrologic model is an approximation of the real phenomena based on thehydrologic cycle, still it is needed to stress that there always exiting discrepancy between model outcome andobserved data, no matter how precise the model is and how perfect the model calibrated. That is the model
predicting limitation underneath the hypothesis and architecture of the model, which is defined as the system
uncertainty in this study.
The system uncertainty can be recognized by evaluating the discrepancy between observed watershed
response series and the model outcome during the process of model parameter calibration. This can be clarified
by the statement written below:~y = f (~x ,θ )

(1)

~
~
where y and x denote outcome and input series of a hydrologic model f(・) respectively, and θ denotes aparameter set. Denoting X and Y are observed watershed input and response series data accordingly and~assuming x = X, then θ can be determined by adjusting or tuning the value of parameters to make the~difference between model outcome y and observed watershed response data Y in an acceptable range
according to some objective function. The procedure of adjusting the value of parameter set is well known as“model calibration”. In this study, least sum of square error (LSE) method is selected as the objective function:
NS LS (θ m ) = ∑ (Yn − f ( X n ,θ m )) 2(2)n =1where n denotes time step of the simulation time series; m denotes the parameter set to be examined.
The discrepancy between observed watershed response and model outcome during the process of calibration
is supposed to be the minimum value by comparison to the possible coming events. This has often been
observed that the goodness-of-fit between observed data and estimated data during calibration is better thanwhat in validation, not to mention in implementation. Under this fact, we can say that the uncertainty occurring
here denotes the predicting limitation of the model. Since this is the best that a model can achieve.
2.2 Entire uncertainty
The discrepancy of the model outcome to the observed data can be shown as below:

Y = f (X ,θ ) + ε(3)where ε denotes system uncertainty mentioned above, that is, the predicting limitation of the model under
perfect calibration. In agreement with this definition, it is clear that the uncertainty came from the process ofcalibration. But the real uncertainty source is supposed to be the data which is used for calibration.
After calibrating the model parameter, the calibrated parameter space will reflects its uncertainty through themodel structure and propagates to the model outcome. This uncertainty can be recognized by examining the282

discrepancy between observed watershed response data and model outcome by using input data and parameter
sets. Also this is the way of most of the researches dealing with parameter sensitivity. This uncertainty isdefined as the entire uncertainty in this study, since actually this is the utmost uncertainty a model could has
under existing input uncertainty.
2.3 Inherent uncertainty
Another categorized uncertainty what we called “inherent uncertainty” is very difficult to be recognized since italways contaminated by the uncertainty of model structure and input data. Inherent uncertainty is different towhat so called parameter uncertainty since the propagation effect of model structure is excluded here
intentionally. Inherent uncertainty represents the sensitivity of parameter space which determined according to
the input uncertainty and reflects to model outcomes. This can be examined by the discrepancy among modeloutcomes derived from different best fit parameter sets. Watershed response data is not used here, whichindicates the model structure uncertainty is eliminated as much as possible.
The distance between entire uncertainty and inherently uncertainty is structure uncertainty. Since theparameter set used for extracting both of the uncertainty is the same. The different is the data used forcomparison: observed data and outcome during calibration. The propagation effect plays an important role
here, which is why the difference between entire uncertainty and inherent uncertainty is called structure
uncertainty in this study.
3UNCERTAINTY QUANTIFICATION

Several indexes are selected to perform uncertainty quantification in this study; the basic form of the index isstatistical second moment and Nash coefficient. The categorized prediction uncertainties defined in previous
section are mathematically described as below:
SDs =∑ ((Q − Q ) − (Q − Q ))obo2(4)bnNashs = 1 −∑ (Q − Q )∑ (Q − Q )o2(5)bo2o(4) and (5) are indices for quantifying system uncertainty, where Qo and Qb indicate the observed watershed
response series and model outcome by using the best fit parameter set, n is the time step of the time series. SDs
quantifies the error variance, the residuals, between observed watershed response and the best fitted simulated
outcomes; while Nashs elucidates the performance of the best fitted simulated outcomes. SD always larger than0, while the range of Nash coefficient is: − ∞ < Nash ≤ 1 .
SDe =∑ ((Q − Q ) − (Q − Q ))oeo2(6)enNashe = 1 −∑ (Q − Q )∑ (Q − Q )
o2(7)

eo

2o(6) and (7) are indices for quantifying entire system, where Qe is the model outcomes acquired by using aparameter set within the whole parameter space and entire rainfall realizations. Consequently, SDe explains thetotal prediction error.
SDi =∑ ((Q − Q ) − (Q − Q ))bebn283e2(8)∑ (Q − Q )∑ (Q − Q )bNashi = 1 −b2e(9)2b(8) and (9) are indices for quantifying inherent system. SDi and Nashi explain the inherent uncertainty because
Qe is calculated with a possible parameter set identified by other model outcomes, thus, the discrepancy between
Qb and Qe indicates the error inevitable in a model formulation.
In order to implement dynamic view of the relationship among system uncertainty, structure uncertainty andparameter uncertainty caused by input data uncertainty, the Nash coefficient is used to formulate a Model
Structure Indicating Index (MSII) and is defined as:
MSII =

Nashi − NasheNashs(10)

The difference between entire and inherent uncertainty denotes the structure uncertainty. This is used as
the numerator in the equation, while system uncertainty is used as a denominator to form a criterion to evaluate
model structure. The index expresses the total variance unexplained by the standard deviation in adimensionless form. The range of MSII is: 0 ≤ MSII < ∞ .
4

ALGORITHM FOR UNCERTAINTY RECOGNITION AND QUANTIFICATION

Instead of sampling the parameter space directly like what GLUE did, the study here generates the parameter set
space by introducing noise item into input data with specified probability distribution. Here Normal distribution
with mean equals to zero and standard deviation from 1.0 to 9.0 (mm/hr) is used to acquire model parameter
space and outcomes under different input uncertainty. For each iteration, 10000 model outcomes for eachspecified input uncertainty were derived from the combination of rainfall series and parameter set generate output
series through the model. The system uncertainty and the prediction ability were identified and recognized bycorresponding parameter set.
5MODEL DESCRIPTION AND PROCESS OF OBSERVED DATATo demonstrate uncertainty identification, Storage Function Model (SFM) proposed by Kimura (Kimura, 1975)
is applied at the Yasu River basin (355km2) in this study. The form of SFM is as:dS= re (t − Tl ) − q, S = kq pdt
re =f × r,r,ifif∑r ≤ R∑r > RSA(11)SA

where S = water storage height; r = rainfall intensity; q = runoff height; t = time step; and Tl = the lag-time.
This model is often used for the flood runoff calculation in a basin with an area of less than five hundred square
kilometers in Japan (Takara, 1985). Parameter p is a constant, commonly the value is 0.6, f is the ratio ofcontribution area of the watershed which generates outflow, RSA is accumulating saturated rainfall. Parameter kis solved by the equation proposed by Nagai et al. (Nagai et al., 1982). A fully functional SFM with parameter
Tl, f, and RSA is called Model 1 in this study. By specifying the value of RSA to 0.0, a poorer-structured model
called Model 2 is manipulated as a contradistinction in this study. Rainfall data was collected from four rainfall
gauging stations inside the Yasu River basin (355km2); they are Yasu, Minakuchi, Kouka and Oogawara. The
average precipitation was obtained by using Thiessen polygon method.

6RESULTSIn Figure 1 and Figure 2, the standard deviation of entire, inherent and system uncertainty of model 1 and 2 are284

shown. The larger magnitude of entire uncertainty indicates a broader range of model outcomes. Which also
indicates a poorer model performance; also this can be examined in Figure 3 and Figure 4: the Nash coefficientof entire, inherent and system uncertainty of Model 1 and Model 2. It is always expected that entire uncertainty
increases while input uncertainty increasing; where the discrepancy between entire and inherent uncertainty is in
the same tendency.
System uncertainty is supposed located in the middle between entire and inherent uncertainty. The range
between entire and inherent uncertainty indicates the goodness of a model structure; that is, the ability of amodel simulates the behavior of the watershed through observed data. The broader the distance is the worse
structure the model has. In Figure 3 and Figure 4, the range between inherent and entire uncertainty of Model 1
is far smaller than Model 2. This can be interpreted that the parameter sensitivity of Model 2 is worse thanModel 1. The reason can be explained by Figure 5 and Figure 6, which show the upper and lower bound ofentire model outcomes and model outcomes during calibration of Model 1 and Model 2 under input uncertainty
4.0 mm/hr. It shows that Model 2 is incapable to simulate the watershed behavior as Model 1 does. Even therange inside upper and lower bound of Model 2 is far larger than Model 1, the true watershed response is out
side the range. This indicates no matter how perfect the calibration is, predicting limitation of Model 2 is worse
than model 1. MSII of Model 1 and Model 2 are illustrated in Figure 7. It shows that the gradient of MSII ofModel 2 increasing more rapidly than Model 1. Which not only reveal the goodness of hydrologic model in astatic view but also reflects the input uncertainty propagates itself through model structure in a dynamic manner.

Figure 1. Standard deviation of model 1.

Figure 2. Standard deviation of model 2.

Figure 3. Nash coefficient of Model 1.

Figure 4. Nash coefficient of Model 2.

Figure 5. Upper and lower bound of Model 1
outcome with input uncertainty 4 mm/hr.

Figure 6. Upper and lower bound of Model 2
outcome with input uncertainty 4 mm/hr.

285

Figure 7.
7

MSII of Model 1 and Model 2.

CONCLUSIONS

A methodology for uncertainty recognition and quantification is proposed which can be used as a tool forhydrologic models quantitative comparison or evaluation. Instead of sampling the parameter space directly likewhat GLUE did, the study here generates the parameter set space by introducing noise item into input data withspecified probability distribution. This reflects the truth that parameter uncertainty came from uncertainty ofdata to hand and the way the model structure responses it. The results show that within increasing input
uncertainty, the distance between entire and inherent uncertainty is also increased. A smaller magnitude of theratio of inherent uncertainty to entire uncertainty indicates the system uncertainty (i.e. the predicting limitation)is larger, which means lacking of the ability to simulate the true watershed behavior.
MSII is proposed to evaluate the goodness of model structure. The results show that a larger value of MSII
indicates a poorer structure of hydrologic model, within increasing input uncertainty the tendency becomes more
apparently. The index can be used as a tool for implementing model quantitative comparison.
8ACKNOWLEDGMENT

This study was supported by Grant-in-Aid for Scientific Research (A)(1) 16206050 (PI: Prof. K. Takeuchi,Yamanashi Univ.) and CREST program of JST (PI: Prof. K. Takara, Kyoto Univ.).
REFERENCES:Kimura, T. (1975). Storage Function Method. Kawanabe Books Co., Ltd., Tokyo (in Japanese).
Nagai, A., Kadoya M., Sugiyama H. and Suzuki K. (1982). "Synthesizing storage function model for flood
runoff analysis." Disaster Prevention Research Institute Annuals, No. 25 B-2, pp. 207-220.
Beven, K. (2001). "How far can we go in distributed hydrological modeling?" Hydrology and Earth System
Science, Vol. 5(1), pp. 1-12.
Freer, J., Beven, K. and Ambroise, B. (1996). "Bayesian estimation of uncertainty in runoff prediction and thevalue of data: An application of the GLUE approach." Water Resources Research, Vol. 32(7), pp.
2161-2173.
Høybye, J. and Rosbjerg, D. (1999). "Effect of input and parameter uncertainties in rainfall-runoff simulation."
Journal of Hydrologic Engineering, Vol. 4(3), pp.214-224.
Kuczera, G. and Parent, E. (1998) "Monte Carlo assessment of parameter uncertainty in conceptual catchment
models: the Metropolis algorithm." Journal of Hydrology, Vol. 211, pp. 69-85.
Singh, V.P. and Woolhiser, D.A. (1976). "Sensitivity of linear and nonlinear surface runoff models to input
errors." Journal of Hydrology, Vol. 29, pp. 243-249.
Singh, V.P. and Woolhiser, D.A. (2002) "Mathematical modeling of watershed hydrology." Journal ofHydrologic Engineering, Vol. 7, pp. 270-292.
Storm, B., Høgh Jensen, K. and Refsgaard, J.C. (1989) "Estimation of catchment rainfall uncertainty and its
influence on runoff prediction." Nordic Hydro., Vol. 19, pp. 77-88.
Takara, K. and Takasao, T. (1985) "A new attempt to evaluate rainfall-runoff models from the viewpoint of
stochastic transformation." Journal of Hydroscience and Hydraulic Engineering, Vol. 3(2), pp. 61-72.

286

contents page