Considerations 

Involved in Evaluating 

Mathematical Modeling of 

Urban Hydrologic Systems

By DAVID R. DAWDY

HYDROLOGIC EFFECTS OF URBAN GROWTH

GEOLOGICAL SURVEY WATER-SUPPLY PAPER 1591-D

UNITED STATES GOVERNMENT PRINTING OFFICE, WASHINGTON : 1969
UNITED STATES DEPARTMENT OF THE INTERIOR 

WALTER J. HICKEL, Secretary

GEOLOGICAL SURVEY 

William T. Pecora, Director

For sale by the Superintendent of Documents, U.S. Government Printing Office 

Washington, D.G. 20402 . Price 20 cents (paper cover)
CONTENTS

Page

Abstract- _ __ _._. _._-___. ____________ Dl

Introduction _. ________ ________ __ 1

Criteria of goodness of fit and the fitting process________________ 4

Effects of errors of data on modeling...___._______._._-_.________ 9

Input errors rainfall._____________________________________ 11

Input errors evapotranspiration________________________________ 12

Output errors streamflow _____________________________ 13

Interpretation of errors_________ _________________________ 14

Sensitivity analysis..___________________________________________ 15

Optimization versus suboptimization _______________________________ 17

Conclusions __________ ________________________________________ 17

References cited _____-_____-_-_-____-____---_._-_---__________ 18

ILLUSTRATIONS

___ Page

FIGURE 1. Diagrams showing comparison of simulation results using 

optimum parameters for each year with results using averaged parameters- _______________________________ D6

2. Diagrams showing comparison of prediction using two sets of 

optimal parameters. Fitting for one is to peak flows, for the 

other, to daily flows_________________________________ 7

3. Resulting flood frequency curves for scatter diagrams of

figure 2______________ _ ________________ 8

4. Diagram of changes in goodness of fit as soil moisture drainage

rate is varied, illustrating nonsymmetry of response________ 16

TABLES

Page

TABLE 1. Qualitative comparison of errors involved in hydrologic 

modeling with analogous errors in standard statistical 

analysis__________________ ________________________ D9

2. The effect of errors in data on the fitting process.___________ 10

m

HYDROLOGIC EFFECTS OF URBAN GROWTH

CONSIDERATIONS INVOLVED IN EVALUATING

MATHEMATICAL MODELING OF URBAN

HYDROLOGIC SYSTEMS

By DAVID E. DAWDY

ABSTRACT

Techniques are available for assessing the performance of mathematical 

models of catchment behavior. These system analysis techniques include optimization methods and use of goodness of fit criteria, error analysis, and sensitivity 

analysis. Further research, however, in model assessment is needed, particularly 

in the application of mathematical modeling to the urban hydrologie environment.

INTRODUCTION

Quantitative hydrology is modeling. The development of hydrology 

is the development of mathematical statements about hydrologie 

processes. Thus flood routing is a mathematical model which describes 

how a flood wave is translated and attenuated as it travels down a 

channel. Whenever a past event is reconstructed or a future event is 

predicted, a model is involved. Interest in the hydrology of the urban 

environment leads to interest in modeling as soon as knowledge is sufficient so that quantitative answers are desired to hydrologie questions.

The field of mathematical modeling can be divided by at least two 

different methods of classification. First is the stochastic-deterministic 

classification. A stochastic model relates input to output statistically. 

This can be by the relation of an input as an independent variable to 

the output as a dependent variable, as in Carter (1961), who related 

change in flood peaks as a result of urbanization to change in lag time. 

It can also be by a statistical simulation of a synthetic streamflow 

trace. Fiering (1967), for instance, computes the statistics of a given 

recorded trace, then uses those statistics to generate "equally likely" 

traces. A deterministic model relates input to output in such a manner 

that, once the input is known, the output is wholly predictable. The 

most often used deterministic models in hydrology are those based on 

the laws of hydraulics, which use equations of continuity and motion.

Dl
D2 HYDROLOGIC EFFECTS OF URBAN GROWTH

Examples are Liggett's (1959) study of open channel flow and Grace 

and Eagleson's (1965) study of similitude for modeling runoff from 

small, impervious areas.

A second division of hydrologic modeling is the analytic-synthetic 

classification. An analytic model usually describes a restricted area of 

hydrology in which the laws governing the process are fairly well 

known and accepted. Given these laws, a means of solution is developed, either in closed form or by finite difference approximations. A 

synthetic model specifies a conceptual relating function, and system 

parameters are identified through the use of input and output data. 

The conceptual model may vary between modelers, if the physical laws 

operating are not well defined or generally agreed upon. Thus, an 

analytic model usually describes a narrow or restricted subsystem in 

hydrology so that the problem is made manageable, whereas a synthetic model can be made as complex and cover as broad an area of 

hydrology as desired. Grace and Eagleson's study is an example of 

the analysis approach, and Crawford and Linsley's (1966) rainfallrunoff model of the synthesis approach.

During the process of model building, the model builder is continually faced with a decision concerning simplicity versus completeness in the model. The simpler the model is, the easier it is to 

understand and use, and probably the cheaper it is to use. However, no 

important element of the system which bears on the process to be 

modeled should be omitted from the model. For example, the use of 

hydraulic equations is familiar to most hydrologists. Because the 

general form of the equations is difficult to solve, simplifications are 

made. Each simplification requires a statement of an assumption, such 

as that steady, uniform flow occurs in a trapezoidal channel. Once 

these assumptions are made the problem is reduced to a manageable 

size and can be solved, but the solution does not apply if the assumptions are violated.

The same problem faces the synthetic model builder as faces the 

analytic model builder, but the decisions are not so easy nor the assumptions so obvious. First the process to be modeled must be broken 

down into subsystems, each subsystem must be judged as to relative 

importance, each must be modeled, and all must be tied together in a 

master program. However, generally accepted concepts such as steady, 

uniform flow or gradually varied flow do not exist. Eather, the question facing a rainfall-runoff model builder might be the relative importance of interception storage as opposed to detention storage, and 

how the two can be differentiated if the two are to be modeled separately. Equations for both must be derived from empirical studies and
EVALUATING MODELING OF HYDROLOGIC SYSTEMS D3

data for neither will be available for most areas of use. Both interception and detention may exist on a basin, yet neither may be necessary 

in a model which aims at a given level of output accuracy. A desire 

for completeness in a model tends to lead toward inclusion of all components which intuitively are known to exist. However, the desire for 

completeness may lead to the inclusion of many parameters which are 

merely curve fitting factors rather than physical parameters describing 

the process they supposedly are modeling. The optimal rule to follow 

would be that of Occam's Eazor, which states that if a simple model 

will suffice, none more complex is necessary. This fails in practice 

unless "suffice" is better defined.

An important problem which faces both the model builder and the 

model user is that concerning the transferability of results. This is 

particularly true in urban hydrology, for little data exist for urban 

watersheds. Practically no data exist for the modeling of the quality 

of urban runoff. Therefore, any model developed must be applicable 

to ungaged areas. Transferability implies that model parameters 

must be derivable from physical measures of the drainage basin. For 

instance, Carter's study requires measures of length, slope, drainage 

area, and degree of urbanization. With these, an estimate of the mean, 

annual flood can be computed. Fiering's streamflow synthesis model 

requires a mean flow, a variance, and a first order serial correlation 

between adjacent flows. From these a synthetic trace can be generated. 

Urbanization studies require that the change in parameters as a result 

of man's influence must be estimated. Carter estimated man's influence 

by using the percentage of impervious area as a parameter. If parameters have true physical significance, such methods may be very 

effective.

The physical significance of the parameters in a typical rainfallrunoff model and their changes as a result of urbanization are discussed by James (1965) in his study of the hydrology of Sacramento, 

Calif. The lack of direct physical significance of the statistical parameters in a streamflow synthesis program is a possible shortcoming. 

Means, variances, and serial correlations must be related to a mappable 

measure. In addition, the statistical parameters used to establish any 

relation must be defined on the basis of a measured period of record. 

Adequate records to define a range for the statistical parameters for 

the urban environment are inadequate or nonexistent. For this reason, 

stochastic models are not considered feasible at this time for modeling urban basins, and therefore are not discussed further. However, 

many of the points which are discussed are applicable both to 

stochastic and deterministic models.
D4 HYDROLOGIC EFFECTS OF URBAN GROWTH

CRITERIA OF GOODNESS OF FIT AND THE 

FITTING PROCESS

Any model must be easily usable and must give satisfactory results. 

While a model builder is concerned with how a model is derived, 

a user is more concerned with what is derived and how well it can 

predict results for his particular problem. "Satisfactory results," however, have no meaning by themselves. There must be some criteria for 

judging goodness of fit.

A best linear predictor for linear systems can be derived by the 

least squares criterion if residual errors are independent, normally 

distributed, and homoscedastic. Even when these conditions are not 

met, some least squares fitting is often used. Yet, even a least squares 

criterion requires that deviations from something must be computed 

in order to have a squared measure to minimize. Eagleson and others 

(1966) explained the rationale for the use of a least squares analysis 

when they stated that the basic problem of linear black box analysis 

is to solve for a meaningful response function when the input and 

output are related by a system that is not truly linear. Eagleson 

obtained an optimal realizable unit hydrograph by using linear programing to solve a Weiner-Hopf formulation for the rainfall-runoff 

system. A solution of the Weiner-Hopf equations gives the leastsquares fit or, in Eagleson's language, minimizes the integral square 

error. The Weiner-Hopf equation, in Eagleson's analysis, minimized 

the squared differences between the simulated and observed total 

streamflow traces. This was an operational decision made in order to 

obtain a hopefully meaningful response function. Any other squared 

error term could have been used, would have given a different WeinerHopf solution, and would have been equally optimal.

Dawdy and Thompson (1967) indicated that the criterion set for 

optimization influences the resulting set of optimal parameters for 

a given model. They indicated that the modeling process can be considered as analogous to a linear programing problem. Eagleson transformed the problem to an explicit formulation of a linear programing 

problem. As Eagleson showed, the goodness of fit criterion in modeling is an objective function, and the model itself, expressed by the 

Weiner-Hopf convolution equations, is a set of constraints.

Sensitivity analysis in linear programing studies the changes in the 

optimal solution as any set of coefficients are varied, including those 

of the objective function. Dawdy and Thompson indicated that in 

their study, three different sets of optimal parameters existed, each 

corresponding to a different objective function. If all three of their 

objective functions were combined into one weighted function, their 

three solutions would be end-member solutions with weights of
EVALUATING MODELING OF HYDROLOGIC SYSTEMS D5

(1,0,0), (0,1,0), and (0,0,1), where the numbers in each set of parentheses represent the relative weights given to each of the three 

objective functions. Each weighting produces a different set of optimal 

parameters. Research into the variation between sets of optimal parameters produced by the use of different objective function weightings 

would give insight into the modeling process and into the interpretation of a given model. Each optimal set of parameters is optimal only 

in terms of its objective function. Most workers have observed this 

seeming paradox in simpler cases of an optimal solution not being a 

unique solution, and it is here merely extended to the general field 

of simulation. Eagleson summarizes the problem when he states, 

"When each storm is analyzed independently, the several unit hydrographs obtained are thus likely to have widely varying geometrical 

properties, and the method of averaging them * * * is not clear." A 

measure of the effect of averaging of parameters on peak estimation 

is shown in figure 1. A rainfall-runoff model was fitted to four different years of record for Arroyo Seco near Pasadena, Calif. Each year 

was fitted separately, and the simulated peaks are shown. In addition, 

the optimum parameters for each year were averaged, and these average parameters were used to simulate the same peaks. The scatter 

increases, particularly for the lower peaks.

There is a direct interaction between the objective function and the 

fitted parameters. Therefore, the choice of the objective function itself 

should be optimal in some sense to the model user. There is no objective method for choosing the dbjective function, however, so that the 

choice of the objective function is a very subjective decision. Research 

in this area is vitally needed.

An example of the effect of the choice of the objective function 

on the fitting process is shown in figure 2. For the same rainfall-runoff 

model, parameters were fitted to a given control period on the basis of 

two different objective functions. The first objective function minimized the sum of squared deviations of simulated discharges from 

observed discharges for peaks, the second for days. The purpose of the 

fitting was to develop a model for estimating peak discharges. The 

goodness of fit was tested by estimating peak discharges for a test 

period on the basis of the two sets of averaged parameters. As can 

be seen, the scatter is about the same for the two sets of simulated peaks 

and is on the same order as the scatter during the control period, as 

shown in figure 1 for the average parameters. However, the parameters 

based on fitting to daily values result in unbiased estimates, whereas 

fitting to peak values resulted in what seems to be a biased estimate. 

This effect in terms of a frequency diagram is shown in figure 3. Part

336-372 69 2
D6 HYDROLOGIC EFFECTS OF URBAN GROWTH

6000

O

z

o

0

u -anno C/) 3000

CC

LU

CL

Hi

LU 

L.

O

CD

O

z 1000

LU

o

cc

2;

0 600

Q

Q

LU

_J

5 300

C/5

100

i

EXPLANATION

o

Optimum parameters for individual

 years of peak criterion 

X

"Average" optimum parameters for

period of peak criterion

y

o

X

0 °

X

° Xx

X O

&

o x

o

X

o

x

o

X

x o

o

o

X

o

X

XX

o

o

o

9

X

o

6

100 300 600 1000 3000 

OBSERVED DISCHARGE, IN CUBIC FEET PER SECOND

6000

FIGURE 1. Comparison of simulation results using optimum parameters for 

each year with results using averaged parameters.

of the hydrologic model became a curve-fitting function and therefore 

did not retain its supposed physical significance when peaks alone 

were used for fitting. Although parameters were "reasonable," the 

water balance was grossly erroneous. This created errors of prediction 

in. the test period.

Sequential fitting methods can use different criteria for fitting different subsets of parameters in a model. A first stage could hold routing 

constants at some first estimate and fit the parameters for water balance. Then the water balance parameters can be held fixed while
EVALUATING MODELING OF HYDROLOGIC SYSTEMS D7

10,000

6000

o

z 

O

o

LJ 

t/5

a: 3000

UJ

1000

600

300

100

EXPLANATION

"Average" optimum parameters-day criterion -

o 

"Average" optimum parameters-peak criterion

100 300 600 1000 3000 6000 

OBSERVED DISCHARGE IN CUBIC FEET PER SECOND

FIGURE 2. Comparison of prediction using two sets of optimal parameters. 

Fitting for one is to peak flows, for the other, to daily flows.

routing parameters are fitted to peaks. However, there is much interaction 'among parameters, and some parameters may influence equally 

both water balance and peak flows. Research in fitting methods can 

go 'hand-in-hand with research in the effects of choice of objective 

function on the optimal fitted solution.

Interactions cannot be eliminated, for they exist in the physical 

system. Their importance in the curve fitting process can be minimized, however, by determining as many parameters outside the model 

as possible. For instance, the Stanford Watershed Model has 20 parameters. Two are based on meteorological data, four on hydrograph 

separation, five are computed from physical measures, three are estimated from empirical tables, and six are fitted. The six fitted parameters all are involved in the loss function, which includes infiltration, 

drainage, and evaporation. Chawford and Linsley (1966) discuss the
D8

1400

1200

Q

z 

o 

o

$ 1000 

cc

LU 

Q_

S 800

u_

O 

CD

0 600

z

u 

o

2? 400

I 

O

w

Q 

200

n

HYDROLOGIC EFFECTS OF URBAN GROWTH

i 

EXPLANATION 

x 

Observed

Simulated "average" day criteria

O 

Simulated "average" peak criteria

O 

0 ,

0° <

O »

.

0 (

o o 

o

* x-» <

x ;

1

,

o °

0 ' 

)

y

< X m 

0.1 0.5 1.0 

RECURRENCE INTERVAL, IN YEARS

5.0

FIGURE 3. Resulting flood frequency curves for scatter diagrams of figure 2.

interactions among the fitted parameters and suggest a combination 

of fitting both sequentially and to selected data to define the parameters as independently as possible.

The determination of parameters a priori eliminates to a degree the 

use of a simulation model as a black box device, and adds a degree of 

grayness. It masks a part of the interaction of parameters because 

some are held constant. However, certain "physical" parameters may 

be indices rather than measures. Resistance coefficients and slope when 

applied to basins as "average values" are examples. If fitted parameters are obtained in order to correlate with measured indices, the interactions plus errors in data and in the model may result in physically 

meaningless values for some or all parameters.

To summarize, fruitful areas of research exist in the development 

of fitting methods, the choice of criteria for judging goodness of fit, 

and the effect of choice of criteria upon fitted parameters.
EVALUATING MODELING OF HYDROLOGIC SYSTEMS D9 

EFFECTS OF ERRORS OF DATA ON MODELING

Errors in data are reflected in errors in the fitted parameters in a 

simulation model. If perfect input data are routed through a perfect 

model, the output produced would agree perfectly with an error-free 

output record. If errors are introduced into the input or output record 

or both, the output will not be exactly reproduced even by a perfect 

model. If a fitting process is used, the parameters will deviate from 

their true values in order to minimize the deviations between the simulated and recorded traces as specified in the objective function. The 

"optimal" set of parameters will now be in error, and the fitted values 

of the objective function will be less than its "true" value.

This process is analogous to statistical least squares analysis. The 

fitted parameters deviate from their population values because of 

random errors in the data. The standard error of estimate is a measure 

of error of reproduction of the fitted data. The standard error of prediction, however, is somewhat greater than the standard error of estimate, for it includes both the measure of lack of fit of the data used to 

fit the model and the measure of error in the fitted parameters. These 

relationships are shown in table 1.

TABLE 1. Qualitative comparison of errors involved in hydrologic modeling with 

analogous errors in standard statistical analysis

Source of error Size of error Statistical analog

Data _____ __ __ __ ___ _

Comparison of measured to simulated 

during period used for fitting. 

Comparison of measured to simulated 

during period not used for fitting.

a-b

a+c

Measurement and

sampling error. 

Standard error of 

estimate. 

Standard error of 

prediction.

If the assumptions of regression theory hold (a linear model with 

normally distributed and homoscedastic errors of the dependent variable) , the error of prediction can be computed from the standard error, 

the deviations of the independent variables from their mean, and the 

error in the coefficients for the independent variables. The assumptions 

seldom hold, however, so that statisticians often resort to split sample 

testing. A similar situation holds for hydrologic simulation, except 

that there is no theory by which to compute the error of prediction. In 

order to present a measure of utility of a model to the potential user, 

the data used to test a model should not include any data used to 

develop the model or its parameters.

Nonlinearity of the hydrologic process precludes any theoretical 

description of the mechanism by which errors in data are transferred
D10 HYDROLOGIC EFFECTS OF URBAN GROWTH

to model parameters and then combined with input errors in the test 

period to produce errors in the simulated streamflow trace. An empirical study for the response for a particular model can be made as in 

table 2. A recorded rainfall trace was assumed error free and routed 

through an optimized set of parameters, which were assumed correct 

values, to obtain a "true" streamflow trace. Then a random error with 

mean zero and standard deviation of 10 percent was applied to all 

rainfall values. These "erroneous" rainfall values were then routed 

through the true model, and the resulting standard error of the simulated streamflow trace computed. An optimization run then was made 

which adjusted the parameters to minimize the standard error. The 

"optimized" set of parameters is shown, along with the resulting 

standard error. The "true" rainfall trace was then routed through the 

optimized parameters, and the standard error computed. Assuming independence of the two souces of error, one in the input data and the 

other in the model parameters, the error of prediction should be approximately equal to the square root of the sum of the squares of the 

two separate estimates. Similar results are shown for random errors 

in the input with 20 percent standard error, and for random errors

TABLE 2. The effect of errors in data on the fitting process

[The model parameters have the following meaning: SWF is suction at the wetted front, KSAT is saturated 

soil permeability, KSW is surface storage coefficient, EVC is evaporation pan coefficient, SMSN is a 

nominal soil-moisture storage, RGF is a multiplicative factor controlling the range of effective SWF, RR 

is a runoff ratio, and D RN is the soil drainage rate]

Parameter value

Parameter

Optimized errors (percent)

True Rainfall

20

Runoff

10

SWF (in.)- __ -

KSAT (in. per 

hr)_____.__.

KSW (hr.)_____ 

EVC__________

SMSN (in.).___ 

RGF_ _ ____-

RR___________

CRN (in. per 

hr)_________

U, Pd 1 (given) _.

pd (fitted) __ .

pD_________.

U, Test 3

3. 6

.063

1.0 

. 56

4. 0 

12. 0

.8

. 020

3. 6

. 063

1.04

.57

4. 02 

11.9

.796

. 018

2. 405(13)

.261(10. 5)

. 105(6. 5)

. 530(14)

3. 8

.06

1.06

.58

3.98 

11.94

.8

. 017

1.45(26)

1.06(19.8)

.41(13) .

2. 40(30) .

3.7

. 063

.98 

.559

4. 04 

12. 12

. 796

. 020

. 063(4. 9)

. 046(4. 0)

3.7

. 061

.98 

. 56

4. 04 

12.21

.796

. 019

. 247(9. 4)

. 191(8. 3)

1 P=True parameters, p=optimized parameters, D = correct data, d=erroneous data, U=value of objective function.

2 Nota bene. First value is sum of squares of difference of natural logarithms of the 18 peaks plus half the 

squares of the 18 storm volumes. The second value converts the first value to an equivalent "percent standard 

error" by SE=aritilog V U/27, and averaging plus and minus percentages.

3 Average of nine separate test runs.
EVALUATING MODELING OF HYDROLOGIC SYSTEMS Dll

in the output of 5 and 10 percent. A further test is then made for each 

case by applying a new set of random errors to the rainfall and routing 

this new set of erroneous data through the "optimized" parameters 

and computing the resulting errors of prediction. These errors also 

are shown in table 1. Such an analysis gives insight into the fitting 

process in simulation models, shows how errors in data are transmitted 

to the model, and gives an empirical measure of the effect of data 

errors on the accuracy of prediction in simulation.

Data errors have different results depending upon the type of data 

considered. All transfer part of their errors into the model parameters, 

but each differently. Each therefore must be considered separately.

INPUT ERRORS RAINFALL

The major source of error results from the rainfall input data. 

Rainfall errors have several sources. There is sampling error as evidenced by variability between measured catches for different designs 

of rain gages, and changes in catch when minor changes are made in 

the configuration of a given rain gage. There are changes with time as 

buildings are built or trees grow nearby, changing wind patterns and 

thus changing catch. There are changes which occur when the gage 

is physically moved, and most long-term records have come from gages 

which had been moved at some time. There is spatial variability of 

rainfall over the basin which a rain gage does not measure because it 

measures point rainfall. Finally, any rainfall record must be considered representative of the basin to be simulated, and there may be 

adjustments made before it is considered representative.

Lumped parameter models of basin response take a rainfall record 

as representative of a basin or subbasin and assume uniform rainfall 

in space. As distributed parameter components or subsystems are included in a given model, input data needs are increased. For example, 

an areally distributed rainfall input becomes necessary in order to be 

compatible with the model, and therefore to realize the benefits of the 

detailed description of the model. The problems of rainfall network 

density for both lumped and distributed parameter modeling have 

been discussed by Amorocho, Brandstetter, and Morgan (1968) and 

Eagleson (1967). Both agree that one or two gages are sufficient to 

estimate mean convective storm precipitation for drainage areas of 

size similar to those found in urban areas. In addition, Amorocho and 

Brandstetter (1966) have studied the problem of rainfall input for 

a distributed system in space. Their conclusions state, "The investigation of questions regarding catchment damping is also essential in order 

to determine the range of departures from the real input field which 

these systems can accept without significant change in the output."
D12 HYDROLOGIC EFFECTS OF URBAN GROWTH

The study of the relation of rainfall network density to accuracy 

still is a fruitful area for research. But errors in data by themselves 

are not of as much importance as the effect of those errors on the decision-making process. The next logical step, as Amorocho and Brandstetter imply, is to study the effect of rainfall errors on simulated 

runoff records. In particular, the use of distributed parameter models 

requires a study of the effect upon simulated streamflow traces of an 

assumption of uniformity of rainfall in space as opposed to the use 

of a spatially varied rainfall.

INPUT ERRORS EVAPOTRANSPIRATION

"Evapotranspiration (ET)" in the main is a depletion from soil 

moisture. An "antecedent precipitation index (API)" is a simplified 

model of the persistence of the effect of past rainfall on future events. 

ET is used in simulation models to construct a better index, or perhaps a "time variable API." ET determines soil moisture depletion, 

and therefore affects the rate of losses of rainfall to soil moisture 

through infiltration. If any concept of a limiting value of minimum 

infiltration is included in a simulation model, major storms of long 

duration and high intensity approach that limiting value, and the 

effect of initial soil-moisture conditions is decreased. Thus streamflow traces for periods of major storms, years of high precipitation, 

and regions of high precipitation should generally be fitted more 

accurately than traces where initial soil moisture conditions are 

critical.

Measured values of soil moisture are seldom available for testing 

depletions from soil moisture through the process of drainage and ET. 

If models of ET loss are to be constructed and then tested by means of 

rainfall-runoff data, the ET model can best be tested in regions where 

soil moisture conditions are critical but variable, or else only selected 

periods of record should be used for model construction. Such periods 

should be selected so as to cover the range of effects of soil moisture on 

storm runoff and peak flows. With such a test, the problems of errors 

in ET can be studied in the manner suggested by Amorocho and 

Brandstetter (1966) for the study of rainfall errors by measuring the 

impact of errors on simulated outputs.

True measures of ET seldom exist. Rather, pan evaporation data is 

collected, pan coefficients are determined, which supposedly adjust 

pan data to a potential evapotranspiration (PET), and then the calculated PET is used as input. Alternatively some PET can be calculated by an empirical (that is, Blaney-Criddle) or semi-empirical
EVALUATING MODELING OF HYDROLOGIC SYSTEMS D13

(Thornthwaite or Penman) equation. Penman (1963) presents various 

theories of "availability of water for evapotranspiration." These different theories represent different modulating functions which convert PET to ET. Errors of output result from errors in PET, 

errors in the conceptual modulating function, and errors of the parameters in the modulating function. Studies of the relative importance 

of these components of error would be worthwhile. In addition, the 

impact of various levels of urbanization on the simulation results will 

give some measure of the degree of attention to these data necessary 

for hydrologic simulation in the urban environment.

OUTPUT ERRORS STREAMFLOW

A discussion of errors of recorded output data centers around errors in stream gaging. The output of interest will depend upon the 

objective function, whether that objective function is stated explicitly 

or is an implicit "let's see how it looks." However, generally the only 

output data available is recorded streamflow. Streamflow data are 

much more accurate than rainfall data because they measure an integrated runoff from the total basin. Most streamflow records are rated 

as "good" by the U.S. Geological Survey (1966), which can be interpreted as meaning that daily mean discharges have a standard 

error of 5 percent (95 percent are within 10 percent). Peak discharges 

are somewhat more inaccurate. Peaks fairly well denned by discharge 

measurements would have a standard error of about 5 percent. By 

"fairly well defined" is meant that the peak flow is no more than twice 

the highest current meter measurement. When not so defined, peak 

flows may be computed by means such as slope-area measurements or 

other indirect methods, and the standard error then may be about 10 

percent.

Errors in the output are transferred to the parameters through the 

fitting process just as are the errors in input. As indicated by the results of table 1, however, output errors are not as critical as input 

errors. Errors in input are magnified because a residual of excess precipitation is used in the routing, and any absolute error in input 

becomes an absolute error in the residual before routing. The generally 

minor amount of surface storage in urban basins does not greatly attenuate input errors. The objective function usually is stated in terms 

of some measure of streamflow, so that proportional errors in measured streamflow are transferred as proportional errors in output.

Errors in input and output impute errors in different parts of the 

model. Random, unbiased errors in input usually are compensated for
D14 HYDROLOGIC EFFECTS OF URBAN GROWTH

by adjustments in the parameters of the loss function, which includes 

infiltration, drainage, interception, and detention. Similar errors in 

output usually are compensated for in the routing function. Biased 

errors in either may act differently. For rainfall, the bias may be an 

error in the adjustment of the recorded record to "average basin precipitation." For streamflW the bias may vary with discharge, with 

little error for the smallest peak and a maximum error for the largest 

peak, if the general slope of the stage-discharge relation used is computed in error.

INTERPRETATION OF ERRORS

The impact of errors upon the simulation process depends in part 

upon whether the error is a random error of a quantity which is measured or whether the error is in an index which is used as an approximation to something which cannot be measured. Data and parameters 

cover the spectrum from streamflow discharge (truly measurable) 

through point rainfall (measurable but used as an index to basin 

rainfall) to PET (not directly measurable and used as an index to 

relate water demand and water availability) to basin slope (some index 

of a varying quantity (Benson, 1959) the effect of which can only 

be estimated in itself). Even grosser indices might be "basin roughness" or "transbasin ground-water seepage" measures.

Errors of indices are errors of approximation in the model. Even 

a "best" value of an index may lead to serious errors in simulation 

when the data are outside the range for which the particular approximation applies. Therefore, errors in indices generate both random errors and errors of approximation. It is errors in indices which usually 

cause outliers, for gross errors usually are the results of poor approximations in a case where the approximation is the controlling factor in 

simulation of a given event.

Errors of approximation become masked in the fitting process. The 

subsystems within the hydrologic cycle are highly interrelated, with 

many interactions among parameters taking place even in the simplest 

models. Therefore the handling of outliers introduces an important 

area of subjectivity in any modeling process. If methods of systems 

identification and systems specification can be developed to separate 

the modeling process into a series of relatively independent problems, a major step forward will result. Sequential fitting methods, as 

mentioned earlier, are a first step in this direction, but much more 

could be done to advance the model development phase if the independent factors could somehow be treated as is done in statistical 

hydrology through eigen vector analysis and synthesis.
EVALUATING MODELING OF HYDROLOGIC SYSTEMS D15 

SENSITIVITY ANALYSIS

Sensitivity analysis studies the effect on the optimal solution of 

changes in the input-output coefficients and in the objective function. 

According to Dantzig (1963),

In many applications, the information thus obtained [through sensitivity 

analysis] is as valuable as the specification of the optimum solution itself. 

Sensitivity analysis is important for several reasons:

(a) Stability of the optimum solution under changes of parameters may 

be critical. For example, using the old optimum solution point, a slight 

variation of a parameter in one direction may result in a large unfavorable difference in the objective function relative to the new 

minimum, while a large variation in the parameter in another direction may result in only a small difference. * * * it may be desirable 

to move away from the optimum solution in order to achieve a solution less likely to require essential modification.

(b) Values of the input-output coefficients, objective function coefficients, 

and/or constraint constants may be to some extent controllable, and 

in this case we want to know the effects which would result from 

changing these values.

<c) Even though the input-output and objective function coefficients and 

constraint constants are not controllable, the estimates for their values 

may be only approximate, making it important to know for what 

ranges of these values the solution is still optimum. If it turns out 

that the optimum solution is extremely sensitive to their values, 

it may become necessary to obtain better estimates.

Sensitivity analyses in hydrologic simulation are not as straightforward as in linear programing. The same principles apply, however. 

Discussions of sensitivity of results to parameter variability are given 

by Crawford and Linsley (1966) and Dawdy and O'Donnell (1965). 

In addition, sensitivity to changes in the objective function (Dawdy 

and Thompson, 1967) were mentioned earlier. A conclusion drawn by 

Dawdy and O'Donnell was that "Any further development of automatic parameter optimization techniques must use some criterion of 

response sensitivity (or its equivalent) in selecting what can be considered adequately optimized parameters. Indeed, the minimization 

of differences from recorded data cannot be the sole criterion in interpreting the fit of any model."

Although all the parameters discussed by Dawdy and O'Donnell 

(1965) apparently were equally sensitive to either positive or negative changes in parameter values, this does not necessarily occur in all 

cases. A case in point is shown in figure 4. A parameter representing 

the rate of drainage of moisture from the soil through percolation to 

the ground-water table was estimated for a simulation run. The sensitivity of the objective function to changes in rate of drainage shows 

that the rate is critical until a certain value is reached. Beyond that
D16 HYDROLOGIC EFFECTS OF URBAN GROWTH

2 50

2 45

« 40

£ 35

UJ 30

H 25

O

LU 20

§ 15

z 10

LU 5 

O

z o

-15 

O £-20

a 25

-50 -40 -30 -20 -10 0 10 20 30 40 50

PERCENT CHANGE IN PARAMETER DRN FROM OPTIMUM VALUE OF 0.05

FIGTTBE 4. Changes in goodness of fit as soil moisture drainage rate is varied,, 

illustrating nonsymmetry of response.

critical drainage rate, the objective function is quite insensitive toincreases in the drainage rate. Therefore, for this model and data it 

is far better to overestimate than to underestimate drainage.

Sensitivity plots for both optimized and computed parameters giveinsight into the modeling process and the interpretation of the physical 

meaning of the parameters in a given model. Sensitivity plots cannot 

be plotted unless the objective function is stated in objective terms.. 

Yet, as Dawdy and Thompson (1967) indicated, the formulation of 

the objective function itself influences the results. At the present time,, 

the use and interpretation of sensitivity analyses is quite subjective. 

A valid field of research which is to date almost untouched is the 

methodology of developing and using sensitivity analyses for comparing different models and for comparing results for a given model 

under differing conditions, as touched upon by Crawford and Linsley 

(1966) and Dawdy and O'Donnell (1965).
EVALUATING MODELING OF HYDROLOGIC SYSTEMS D17 

OPTIMIZATION VERSUS SUBOPTIMIZATION

Up to this point, all discussion has centered upon hydrology as a 

system. Optimization has been in terms of errors in simulation of 

hydrologic data. In fact, hydrology is a subsystem, and the discussion 

has concerned suboptimizatkm. Hydrology is one input into a decision 

process concerning resources development. In optimization of the decision process, accuracy of hydrologic simulation has both costs and 

benefits with reference to the ultimate development scheme. Costs are 

incurred in gathering data of a certain type at a certain level of accuracy for a given length of time. Benefits result from better decisions 

which result from better data. Discussion of this broader systems 

study of the marginal cost and marginal worth of hydrologic data is 

beyond the scope of the present study. Nevertheless the broader problem must be considered. It also presents many areas where research 

is needed.

CONCLUSIONS

Hydrologic simulation today is half science and half art. It is an 

art to the extent that subjective decisions enter into the modeling 

process and its assessment. Kesearch is needed to move the field closer 

toward the area of science by developing and using the necessary systems-analysis techniques. Measures of error are necessary in order to 

judge between models, whether on the part of the model builder or the 

model user. In addition, measures of error must usually be stated in 

some form of an objective function in order to us the tools of systems 

analysis.

Fruitful areas of research are:

1. The construction of meaningful objective functions and the interplay between objective function and simulation results. The use 

of objective functions for the disparate aims of fitting and predicting within a single model and of comparing between models 

should be considered.

2. The development of tools for sensitivity analyses and of methods 

for using sensitivity analyses for developing and using simulation models.

3. The development of mathematical tools for fitting the independent 

parts of hydrology. This would be in the sense of eigen vector 

analysis, in which the independent factors are not necessarily the 

original variables nor subsets of the original variables, but some 

combination of the original variables.
D18 HYDROLOGIC EFFECTS OF URBAN GROWTH

4. The determination of guidelines for error analysis in hydrologic 

simulations. Hydrologic models are nonlinear, so that standard 

linear theory does not apply. Errors in data are transferred into 

fitted parameters. These errors in fitted parameters plus those 

in computed parameters are transferred into predictions. Little 

is understood of this mechanism.

5. The comparative effects of input errors and output errors on simulation results so that some concept of network density can be 

applied to urban hydrologic data in terms of the end results of 

hydrologic prediction through simulation.

REFERENCES CITED

Amorocho, Jaime, and Brandstetter, Albin, 1966, Characterization of gage level 

precipitation patterns: Presented at the Pacific Southwest Regional meeting, Am. Geophys. Union, San Jose, Calif., February 1966.

Amorocho, Jaime, Brandstetter, Albin, and Morgan, Don, 1968, the Effects of 

density of recording raingauge networks on the description of precipitation patterns: Internat. Assoc. Sci. Hydrology Pub. no. 78, p. 189-202.

Benson, M. A., 1959, Channel-slope factor in flood-frequency analysis: Am. Soc. 

Civil Engineers, separate no. 1994, 9 p.

Carter, R. W., 1961, Magnitude and frequency of floods in suburban areas in 

Short papers in the geologic and hydrologic sciences: U.S. Geol. SurveyProf. Paper 424-B, p. B9-B11.

Crawford, N. H, and Linsley, R. K., 1966, Digital simulation in hydrology: 

Stanford Watershed Model IV, Dept. of Civil Eng. Tech. Rept. no. 39, Stanford University.

Dantzig, G. B., 1963, Linear programming and extensions: The Rand Corp., and 

University of California, Berkeley, 632 p.

Dawdy, D. R., and O'Donnell, Terence, 1965, Mathematical models of catchment behavior: Am. Soc. Civil Engineers Proc. Paper 4410, HY4, p. 123-137.

Dawdy, D. R., and Thompson, T. H., 1967, Digital computer simulation in 

hydrology: Jour. Am. Water Resources Assn., v. 59, no. 6. p. 685-688.

Eagleson, P. S., 1967, Optimum density of rainfall networks: Water Resources 

Research, v. 3, no. 4, p. 1021-1034.

Eagleson, P. S., Mejia-E, Bicardo, and March, Frederic, 1966, Computation of 

optimum realizable unit hydrographs: Water Resources Research, v. 2, no. 4> 

p. 755-764.

Tiering, M. B., 1967, Streamflow Synthesis: Harvard Univ. Press.

Grace, R. G., and Eagleson, P. S., 1965, Similarity criteria in the surface runoff 

process, Dept. of Civil Eng., Hydrodynamics Laboratory Rept. no. 77, M.I.T.

James, L. D., 1965, Using a digital computer to estimate the effects of urban, 

development on flood peaks: Water Resources Research v. 1, no. 2, p. 223-234..

Liggett, J. G., 1959, Unsteady open channel flow with lateral inflow: Dept. of 

Civil Eng. Tech. Rept. no. 2, Stanford University.

Penman, H. L., 1963, Vegetation and hydrology: Tech. Comm. no. 53, Commonwealth Bureau of Soils, Commonwealth Agricultural Bureaux,, Farnham 

Royal, Bucks, England, 124 p.

U.S. Geological Survey, 1966, Water resources data for California, Part 1: Surface Water Records.

U.S. GOVERNMENT PRINTING OFFICE: I96»