Number range CHAPTERNWS Hydrologic Model Calibration

Research and Development

Michael Smith, Victor Koren, Seann Reed, Ziya Zhang, Fekadu Moreda,

Richard Anderson, Vadim Kuzmin, Zhengtao Cui, Eric Anderson

Hydrology Laboratory

Office of Hydrologic Development

November 13, 2023

![](media/image1.wmf){width="4.918055555555555in"
height="3.688888888888889in"}

Executive Summary

The National Weather Service Hydrology Laboratory (NWS HL) has a rich
history of hydrologic model calibration research. This research program
combined extensive in-house efforts with the pioneering work of the
University of Arizona. NWS efforts in the calibration arena are well
known as evidenced by the recent monogram 'Calibration of Watershed
Models' edited by HL employee Qingyun Duan and others.

HL calibration research has led to numerous tools, applications and
guidelines. On a regular basis, HL leads calibration workshops for
personnel from the River Forecast Centers (RFCs). These are often
attended by scientists and students from the University of Arizona,
resulting in a fruitful two-way transfer of knowledge and expertise.
Examples of this transfer include the development of the Mult-step
Automatic Calibration Scheme (MACS) developed by the University of
Arizona that uses automatic optimization in three phases to mimic the
manual calibration process. Other noteworthy accomplishments are the
production of a video series on calibration of the Sacramento Soil
Moisture Accounting Model (SAC-SMA), the main rainfall-runoff model used
for NWS river forecasting. This video series, created through a
collaboration between HL and Konstantine Georgakakos of the Hydrologic
Research Center, has been delivered to and used extensively at the RFCs.
In addition, this video series has been used internationally in
countries that have adopted the NWS River Forecast System (NWSRFS).

A significant advance has been the publication of a 340 page manual on
all phases of model calibration, from data preparation to implementation
of calibration results for operational forecasting.

We believe that manual, guided trial-and-error procedures have a
prominent place in the calibration process. However, we are actively
engaged in research into automatic methods including single and
multi-objective optimization techniques. An active area of work is the
development of a local search procedure called Step-wise Line Search
(SLS). The use of this procedure and a priori estimates of SAC-SMA
parameters results in parameters that are as good as those derived from
the well-known global search Shuffled Complex Evolution procedure (SCE).
The benefit of the SLS-based procedure is that it requires only a
fraction of the number of function evaluations that SCE needs. Thus,
this procedure has promise not only for lumped modeling but also for
distributed modeling where the number of computational cells would
render a procedure like SCE computationally infeasible.

Contents

1\. Introduction 4

2\. Purpose and Scope 4

3\. Relevant Web Sites 4

4\. Research and Development Strategy 4

5\. Internal HL R&D 8

Manual calibration ICP 8

Calibration Assistance Program 8

Derivation of *A Priori* Parameter Estimates 10

Quality Control of Data for Calibration 10

Calibration Training Manual 10

Design of Enhanced Calibration System 11

Automatic Calibration 11

6\. Collaborative Research and Development 11

7\. Tools, Procedures and Guidelines delivered to RFCs 12

8.  Recommendations 12

9.  References 13

Appendix 1 Publications and Presentations from Calibration R&D 15

Appendix 2 What will speed-up model calibration? 16

Appendix 3 Calibration Assistance Program Summary 18

1.  Introduction

Figure 1 shows the domain of each of the 13 River Forecast Centers
(RFCs) of the NWS. Within these areas, the NWS generates daily river
forecasts at over 4,000 points. Much finer resolution forecasts are
provided as required by specific conditions. To provide accurate
forecasts, the models that produce these forecasts must be calibrated.
The scale of the NWS hydrologic forecasting mission has immense
implications for the process of model calibration Gupta et al. (2003).
To meet the demands of calibrating nationally-implemented hydrologic
models, a comprehensive calibration system has been developed. As shown
in Figure 2, the Calibration System (CS) is a major component of the
NWSRFS. It contains procedures for data analysis and parameter
calibration using manual trial-and-error or automatic calibration
methods (Smith et al., 2003).

2.  Purpose and scope.

The purpose of this document is to provide an overview of the research
and development of hydrologic model calibration, including parameter
estimation. Such models as the NWS operational snow model (Snow-17), the
Sacramento Soil Moisture Accounting model (SAC-SMA), and hydrologic
routing components are included here. Also, this document overlaps the
calibration of kinematic wave channel routing parameters for distributed
modeling. However, this document does not cover the calibration of NWS
dynamic wave channel routing models.

While past efforts regarding hydrologic model calibration will be
discussed, the emphasis of this document is on current and future
activities and strategies.

3.  Relevant web sites

The reader is directed toward items on the following web page for
additional information.

http://www.nws.noaa.gov/oh/hrl/calb/calbmain.htm

4.  R&D strategy

Figure 3 presents the strategy for hydrologic model calibration R&D. As
with other areas of research, HL places high value on a combination of
internal and extramural research. Indeed, a long collaborative research
relationship between HL and the University of Arizona has resulted in a
successful two-way exchange of knowledge and experience. On the one
hand, the University of Arizona developed the SCE method (Duan et al.,
1993), and then the developer himself was employed in HL for 12 years.
SCE is perhaps the most widely used of the automatic optimization
procedures available in the NWSRFS Calibration System. On the other
hand, understanding of the NWS manual calibration procedures led to the
development of the Multi-step Automatic Calibration Scheme (MACS) (Hogue
et al., 2003). Significant internal efforts have been aimed at the
development of manual and automatic calibration procedures and tools.
Moreover, HL scientists devote considerable time to developing
calibration techniques for any new models.

![](media/image2.wmf)

Figure 1. Boundaries of NWS River Forecast Center domains (Alaska
Pacific RFC not shown)

![](media/image3.wmf){width="7.604861111111111in"
height="5.729166666666667in"}

Figure 2. Components of the NWSRFS (Source: Smith et al. 2003)

<table>
<colgroup>
<col style="width: 16%" />
<col style="width: 5%" />
<col style="width: 5%" />
<col style="width: 5%" />
<col style="width: 5%" />
<col style="width: 5%" />
<col style="width: 5%" />
<col style="width: 5%" />
<col style="width: 5%" />
<col style="width: 5%" />
<col style="width: 5%" />
<col style="width: 5%" />
<col style="width: 5%" />
<col style="width: 5%" />
<col style="width: 5%" />
</colgroup>
<tbody>
<tr class="odd">
<td>Topic Year</td>
<td>‘94</td>
<td>‘95</td>
<td>‘96</td>
<td>‘97</td>
<td>‘98</td>
<td>‘99</td>
<td>‘00</td>
<td>‘01</td>
<td>‘02</td>
<td>‘03</td>
<td>‘04</td>
<td>‘05</td>
<td>‘06</td>
<td></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td><p>HL Internal</p>
<p>R&amp;D</p></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Interactive Calibration Program</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Calibration Assistance Program</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>A-Priori parameter derivation</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td><p>Calb. data quality control:</p>
<p>Double mass analysis</p></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Calibration Manual</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Design new calb. system</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Constrained auto. opt</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Auto calb. Step-wise line search</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Optimization of channel routing via VAR</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Collaborative R&amp;D</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>U. Arizona automatic optimzation</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>HRC Calibration Training video</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>FY04 Competitive Research Program</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
</tbody>
</table>

Figure 3. Major plan elements for hydrologic model calibration research
and development

5\. HL Internal Research and Development

[Manual-Expert Calibration]{.underline}

Gupta et al. (2003) provide an excellent overview of the history of
automatic calibration procedures. Moreover, they also clearly discuss
their deficiencies, stating that:

'.... research conducted during the past two and a half decades has
revealed that the traditional automatic calibration methods suffer from
a number of serious conceptual and practical weaknesses'.

Indeed, Schaake (2003) states the widely recognized fact that automatic
calibration using a single objective function (as in most available
automatic calibration schemes) does not adequately measure the ways in
which a model fails to fit the real data. In light of these current
deficiencies, we maintain that manual trial-and-error (hereafter called
Manual-Expert calibration after Gupta et al. (2003)) approaches embody
the preferred strategy for model calibration in the NWS. Moreover, many
RFCs advocate for Manual-Expert calibration, believing that this process
is a very beneficial experience in the development of a hydrologic
forecaster (Smith et al., 2003).

To support Manual-Expert calibration, HL has developed a number of
visualization tools. A major advance in manual calibration was the
development of the Interactive Calibration Program (ICP) in the mid
1990's shown in Figure 4. Smith et al. (2003) present an overview of the
capabilities of ICP. ICP is extremely powerful, able to display multiple
time series of input forcings, model states, and output values. Various
analysis tools are also in ICP to aid the calibration process. ICP is
also heavily used in many other areas of HL research and development:
snow, frozen ground, and distributed modeling. Indeed, ICP was the main
tool to display simulations at the August, 2002 meeting of the
participants of the international Distributed Model Intercomparison
Project (DMIP) (Smith et al., 2004).

Unfortunately, ICP development has largely stagnated as a result of the
untimely death of the main developer in 1999. After this, ICP
enhancements have been identified (Anderson, 2003;
<http://www.nws.noaa.gov/oh/hrl/calb/ICP_Recom/index.htm>) but have
never received a high enough priority in the HL work queue to be
implemented. A very modest effort has been started in FY04 to define the
functional requirements of ICP in order to perform a re-software
engineering of the entire code. Depending on the availability of
funding, the re-engineering process will be initiated so that ICP is
updated.

[Calibration Assistance Program]{.underline}

Number range CHAPTERThe HL-developed Calibration Assistance Program
(CAP) (see Reed, 2002; and
<http://www.nws.noaa.gov/oh/hrl/calb/esrihtmlfinal/p1082.htm>) contains
data sets and tools that are primarily used for the analysis of
mountainous areas. CAP is a *national* ArcView GIS- based suite of tools
that facilitates the derivation of basin sub-divisions, model
parameters, potential evaporation estimates, and mean areal estimates of
precipitation and temperature. Moreover, CAP also contains satellite
snow cover maps that can be used in conjunction with observed streamflow
hydrographs and area elevation curves to identify different regimes that
occur over the watershed. CAP also contains national vegetation maps to
help identify such regimes. Appendix 3 provides more details about the
functions available in CAP.

![](media/image1.wmf){width="6.334027777777778in"
height="4.759722222222222in"}

Figure 4. Main ICP display. Pane 1 shows the computed frost index for
frozen ground computations. Pane 2 shows the rain and snow melt
timeseries. Pane 3 displays the runoff components generated by the
SAC-SMA model. Pane 4 shows the states of the 5 main storage reservoirs.
Pane 5 shows the incoming rain-plus-snowmelt each time step. Pane 6
shows hydrograph plots for the period from January, 1973 to March, 1973.

[Derivation of A Priori Hydrologic Model Parameters]{.underline}

We believe that research into the derivation of a priori estimates of
parameters for hydrologic models is an integral part of the calibration
process. Meaningful initial estimates of model parameters simplify the
Manual-Expert calibration process. Traditionally, hydrograph analysis
techniques have been used to derive initial estimates of some model
parameters. Pioneering research by Koren et al. (2000) resulted in
promising estimates of all the parameters for the Sacramento Soil
Moisture Accounting model (SAC-SMA). This method used the state-wide
coarse resolution soils information (STATSGO) available from the
National Resources Conservation Service (NRCS). The parameters of Koren
et al. (2000) have been made available to all the RFCs via the CAP. Duan
et al. (2001) used these parameters for land surface models. Recent work
by Anderson et al. (2003) investigates the utility of NRCS fine-scale
county level soils data (SSURGO) in the derivation of SAC-SMA
parameters. This research should tell us whether fine-scale soils
information is needed or not. Complimenting the work with the SAC-SMA
model parameters, Moreda et al. (2003) developed procedures to derive a
priori estimates of parameters for a continuous antecedent precipitation
index model (API) being used in a distributed format.

Significant effort has been directed at the derivation and calibration
of channel routing parameters for the kinematic wave component of the
distributed model developed by HL (Reed et al., 2002; see also
PowerPoint file: AGU_spring_2002_Reed). The premise of this research is
that basin-outlet channel geometry information and outlet discharge
measurements can be combined with stream channel morphological
relationships to estimate routing parameters at grid cell locations in
the basin.

[Quality Control of Calibration Data: Consistency
Corrections]{.underline}

Smith et al. (2003) outline the reasons for quality control of the data
used for model calibration. As shown in Figure 3, we are analyzing the
effects of performing and not performing one of the recommended quality
control procedures called double mass analysis of data consistency
(Linsley et al., 1982; see page 70). This study should help us determine
the sensitivity of hydrologic simulations and forecasts to errors in
calibration data.

[Calibration Manual]{.underline}

While not strictly a research effort, a major accomplishment in 2002 was
the development and publication of a comprehensive manual covering all
phases of model calibration. (Anderson, 2002). (see
<http://www.nws.noaa.gov/oh/hrl/calb/calibration1102/main.htm>. Topics
covered in this 350 page compendium range from analysis of historical
data to the incorporation of calibration results into the NWS
operational system. Also discussed is the very important yet often
neglected concept of examining and maintaining consistency between the
historical data used for model calibration and the real-time observed
data used for operational forecasting. Such analyses are critical in
that studies have shown that a bias of as little as 3^o^F can have major
effects on the timing of the onset of snowmelt (Anderson, 2002). This
manual has been made available online and was also delivered to the
University of Arizona.

[Design of Enhanced Calibration System]{.underline}

As shown in Figure 2, the Calibration System represents about one-third
of the functionality of the entire NWSRFS. It was designed to provide a
seamless progression from model calibration to use of the model for
operational forecasting. In addition, it was designed as an R&D
environment for new NWSRFS models and components. In spite of its
overall importance, very few enhancements have been made to the
Calibration System. The preeminent need according to Smith et al. (2003)
is to more efficiently link the functions performed by the various
disparate programs in the Calibration System. To this end, a list of
functional requirements has been prepared (Anderson, 2003b) (document
available online at
<http://www.nws.noaa.gov/oh/hrl/calb/EnhancedCalb/index.htm>). However,
as with ICP, the effort to enhance the Calibration System has never
achieved high enough priority to have resources directed to this task.

[Automatic Calibration]{.underline}

As stated earlier, HL is actively engaged in automatic optimization
research. Recent efforts have focused on two areas. First, a procedure
has been developed to utilize a priori estimates of the SAC-SMA model to
constrain the automatic optimization process when calibrating a number
of neighboring basins (Koren et al., 2003). The idea here is that the a
priori estimates can be used to guide the optimization process so that
the final model parameters have a logical spatial consistency. This work
is continuing in the sense that research into a priori estimates of
model parameters is ongoing as discussed above.

HL and the NWS field offices have used SCE since its development.
Recently, research was initiated to develop a new procedure to search
the error surface for minima (Kuzmin et al. 2004; see also file:
Kuzmin_presentation_04.ppt). This procedure is called the Step-wise Line
Search (SLS) method. The premise of this effort is that given reasonable
estimates of the initial parameters of a model, a global search
procedure is not needed. Rather, it is sufficient to have a local search
procedure. Research to date has shown the SLS method used with the a
priori parameters of Koren et al. (2002) achieves parameters that are
equivalent to those derived using the SCE approach. However, the SLS
method requires only a small fraction of the number of function
evaluations required by SCE. Given this success, we see the greatest
utility of the SLS approach being the calibration of a distributed
model. The use of a global search algorithm such as SCE for distributed
modeling applications is computationally infeasible.

As part of this work, HL is investigating new objective functions for
use with automatic optimization. A multi-scale objective function
evaluating monthly, weekly, daily, and hourly flow errors is being
tested. Results to date indicate that a smoother objective function
surface is produced.

6\. Collaborative Research and Development

As shown in Figure 3, HL has a long and productive history of
collaboration with leading institutions. For many years, HL has
supported automatic optimization research at the University of Arizona.
A significant two-way exchange of knowledge has occurred during this
period. HL adopted the SCE algorithm and even employed its developer for
over 12 years. On the other hand, the MACs procedure discussed earlier
was developed in part from U. of Arizona's participation in HL's
calibration training workshops conducted at RFCs. Further evidence of
this transfer can be seen in the references to HL Manual-Expert
calibration techniques in several peer-reviewed papers authored by U. of
Arizona scientists (e.g., Gupta et al., 2003; Boyle et al., 2000). HL
has also supported the research into the calibration of semi-distributed
models (e.g., Boyle et al., 2001; Khodatalab et al., 2004).

A practical yet significant achievement was the development of a
calibration training video through a collaboration between the
Hydrologic Research Center (HRC, 1999) and HL. This effort was sponsored
and funded by the World Meteorological Organization (WMO). This video
series covers in great depth the calibration of the SAC-SMA model, which
is the primary rainfall-runoff model used at NWS RFCs. This video series
has been delivered to all NWS RFC offices and has been used in many
institutions internationally.

7\. Tools, procedures, and guidelines delivered to RFCs.

HL's combined internal and collaborative R&D efforts have resulted in
significant and practical tools, procedures and guidelines delivered to
the RFCs. Table 1 presents the major items that have been delivered.

Table 1. Summary of major tools, procedures, and guidelines delivered to
NWS RFCs.

  ----------------- ------------------------ -----------------------------
  Name              Description              Status

  Interactive       Powerful GUI with many   Enhancements designed, held
  Calibration       display and analysis     up by low priority
  Program (ICP)     features                 

  Calibration       4-part video series on   Delivered to all RFCs;
  Training Video    calibration of SAC-SMA   distributed internationally
                    model.                   

  Calibration       ArcView-based system     Available at all 13 River
  Assistance        providing numerous       Forecast Centers
  Program (CAP)                              

  MACS procedure    Multi-step automatic     Presented at calibration
                    calibration Scheme by U. workshops. Part of
                    Arizona                  collaborative work between
                                             CBRFC and U. AZ

  Interactive       GUI tool for quality     Delivered to all RFCs and
  Double Mass       control of calibration   officially supported.
  Analysis          data.                    
  Technique (IDMA)                           

  Calibration       350-page comprehensive   Available on line; delivered
  Training Manual   guide to all phases of   to all RFCs and U. of
                    model calibration        Arizona.
  ----------------- ------------------------ -----------------------------

8\. Recommendations

Given the scope of the NWS forecasting mission and the subsequent
immense implications for hydrologic model calibration as part of the
AHPS initiative, we recommend that funding be allocated so that that the
Calibration System (especially ICP) can be upgraded (Smith et al, 2003).
More efficient links between the many disparate programs in the
Calibration System are needed. This can be most effectively achieved
through a re-design of the entire system. Such a re-designed system
would also be easily enhanced as new procedures are developed.

9\. References

Anderson, E.A., 2003a. Number range CHAPTERInteractive Calibration
Program-Suggested Changes and Enhancements, Hydrology Lab internal
report. January. Available online at:
<http://www.nws.noaa.gov/oh/hrl/calb/ICP_Recom/index.htm>

Anderson, E.A., 2003b. Functional Requirements for an Enhanced
Calibration System, Hydrology Lab internal report. May. Available online
at:

<http://www.nws.noaa.gov/oh/hrl/calb/EnhancedCalb/index.htm>

Anderson, E.A., 2002. \'Calibration of Conceptual Models for Use in
River Forecasting\', August, 350pp. Hydrology Lab report. Available
online at:

<http://www.nws.noaa.gov/oh/hrl/calb/calibration1102/main.htm>

Anderson, R., Koren, V., Smith, M., and Seo, D.-J., 2003. On the
importance of linear vs. nonlinear response in estimating base flow
features for ungauged catchments in Southern Texas. Proceedings of the
Fall Meeting of the AGU, December.

Boyle, D.P., Gupta, H.V., Sorooshian, S., Koren, V., Zhang, Z., and
Smith, M., 2001. Toward improved streamflow forecasts: Value of
semi-distributed modeling. Water Resources Rearch, Vol. 37, No. 11,
2749-2759.

Boyle, D.P., Gupta, H.V., and Sorooshian, S., 2000. Toward improved
calibration of hydrologic models: Combining the strengths of manual and
automatic methods. Water Resources Research, Vol. 36, No. 12, 3663-3674.

Duan, Q.-Y, 2003. Global optimization for watershed model calibration.
In: Duan, Q., Gupta, H.V., Sorooshian, S., Rousseau, A.N., Turcotte, R.,
(Editors.) Calibration of Watershed Models, Water Science and
Application 6, American Geophysical Union, Wash. D.C., 89-112.

Duan, Q., Schaake, J., and Koren, V., 2001. A priori estimation of land
surface model parameters, in Laskhmi, V. et al., (Eds.), Land Surface
Hydrology, Meteorology, and Climate: Observations and Modeling, Water
Science and Applications 3, AGU, Washington, D.C., 77-94.

Duan, Q., Gupta, V.K., and Sorooshian, S., 1993. A shuffled complex
evolution approach for effective and efficient global optimization,
Journal of Optimization Theory and its Applications, Vol. 76, No. 3,
501-521.

Gupta, H.V., Sorooshian, S. Hogue, T.S., and Boyle, D.P., 2003. Advances
in automatic calibration of watershed models. In: Duan, Q., Gupta, H.V.,
Sorooshian, S., Rousseau, A.N., Turcotte, R., (Editors.) Calibration of
Watershed Models, Water Science and Application 6, American Geophysical
Union, Wash. D.C., 9-28.

Hogue, T.S., Gupta, H.V, Sorooshian, S., Tomkins, C.D., 2003. A
multi-step automatic calibration scheme for watershed models. In: Duan,
Q., Gupta, H.V., Sorooshian, S., Rousseau, A.N., Turcotte, R.,
(Editors.) Calibration of Watershed Models, Water Science and
Application 6, American Geophysical Union, Wash. D.C., 165-174.

Hydrologic Research Center, (producer), 1999. Calibration of the
Sacramento Soil Moisture Accounting Model, Demonstration of an
Interactive Calibration Approach. NOAA Video Series: VHS Video (5hrs and
15min) and Companion Notebook (200pp). National Oceanic and Atmospheric
Administration, National Weather Service, Office of Hydrologic
Development, Silver Spring, MD., May. Copies available upon request.

Khodatalab, N., Gupta, H., Wagener, T., and Sorooshian, S., 2004.
Calibration of a semi-distributed hydrologic model for streamflow
estimation along a river system. Journal of Hydrology Special Issue on
DMIP, pending final acceptance by Chief Editor.

Koren, V.I., Smith, M., and Duan, Q.-Y, 2003. Use of a priori parameter
estimates in the derivation of spatially consistent parameter sets of
rainfall-runoff models. In: Duan, Q., Gupta, H.V., Sorooshian, S.,
Rousseau, A.N., Turcotte, R., (Editors.) Calibration of Watershed
Models, Water Science and Application 6, American Geophysical Union,
Wash. D.C., 239-254

Koren, V.I., Smith, M., Wang, D., Zhang, Z. 2000. Use of Soil Property
Data in the Derivation of Conceptual Rainfall-Runoff Model Parameters
80th Annual Meeting of the AMS, Long Beach, Ca. January.

Kuzmin, V., Koren, V.I., and Seo, D.-J., 2004**.** Calibration of the
Sacramento model: Stepwise line search. In preparation.

Linsley, R.K., Kohler, M., and Paulhus, J.L.H., 1982. Hydrology for
Engineers. McGraw-Hill, New York, 508 pp.

Number range CHAPTERMoreda, F.G., Zhang, Z., Koren, V., Reed, S., and
Smith, M., 2003. On the parameterization of distributed models. Abstract
HW08/08A/C28-005, XXIII General Assembly of the International Union of
Geodesy and Geophysics-IUGG 2003 Sapporo, Japan, June 30-July 11. Full
paper to be submitted to a special issue of the Journal of Hydrology
dedicated to MOPEX.

Reed, S.M., Koren, V., Zhang, Z., and Smith, M., 2002. Assessment of
Distributed Hydrologic Routing Parameter Estimates used in NWS-HL-RMS.
Spring Meeting of the AGU, Washington, D.C., paper H51B-12.

Smith, M.B. and coauthors, 2004. The distributed model Intercomparison
project (DMIP): Motivation and experiment design. Journal of Hydrology
Special Issue on DMIP, pending final acceptance by Chief Editor.

Smith, M.B., Laurine, D.P., Koren, V.I., Reed, S.M., and Zhang, Z**.**,
2003. Hydrologic Model Calibration in the National Weather Service. In:
Duan, Q., Gupta, H.V., Sorooshian, S., Rousseau, A.N., Turcotte, R.,
(Editors.) Calibration of Watershed Models, Water Science and
Application 6, American Geophysical Union, Wash. D.C., 133-152.

**Appendix 1**

**Papers and Publications from Hydrologic Model Calibration Research**

[Peer-Reviewed Papers in Scientific Journals (HL scientists in
bold)]{.underline}

**Duan, Q.-Y**, 2003. Global optimization for watershed model
calibration. In: Duan, Q., Gupta, H.V., Sorooshian, S., Rousseau, A.N.,
Turcotte, R., (Editors.) Calibration of Watershed Models, Water Science
and Application 6, American Geophysical Union, Wash. D.C., 89-112.

**Koren, V.I., Smith, M., and Duan, Q.-Y**, 2003. Use of a priori
parameter estimates in the derivation of spatially consistent parameter
sets of rainfall-runoff models. In: Duan, Q., Gupta, H.V., Sorooshian,
S., Rousseau, A.N., Turcotte, R., (Editors.) Calibration of Watershed
Models, Water Science and Application 6, American Geophysical Union,
Wash. D.C., 239-254

**Kuzmin, V., Koren, V.I., and Seo, D.-J., 2004.** Calibration of the
Sacramento model: Stepwise line search. In preparation.

**Schaake, J.C**., 2003. Introduction. In: Duan, Q., Gupta, H.V.,
Sorooshian, S., Rousseau, A.N., Turcotte, R., (Editors.) Calibration of
Watershed Models, Water Science and Application 6, American Geophysical
Union, Wash. D.C., 1-8.

**Seo, D**.-**J, Koren, V., and Reed, S.,** 2003. Improving a priori
estimates of hydraulic parameters in a distributed routing model via
Variational assimilation of long-term streamflow data. Weather Radar
Information and Distributed Hydrological Modeling, IAHS Publication no.
282., Eds.: Tachikawa et al., proceedings of Symposium HS03 held during
the IUGG 2003, Sapporo, Japan, June 30-July 11, 109-113.

**Smith, M.B., Laurine, D.P., Koren, V.I., Reed, S.M., and Zhang**,
Z**.**, 2003. Hydrologic Model Calibration in the National Weather
Service. In: Duan, Q., Gupta, H.V., Sorooshian, S., Rousseau, A.N.,
Turcotte, R., (Editors.) Calibration of Watershed Models, Water Science
and Application 6, American Geophysical Union, Wash. D.C., 133-152.

[Presentations at conferences]{.underline}

1\. **Reed, S.M, King, S., Koren, V., Smith, M., Zhang, Z., and Wang,
D**., 2001. Parameterization Assistance for NWS Hydrology Models Using
ArcView. ESRI Users Conference.

Appendix 2

What will Speed Up the Calibration Process?

Mike Smith and Janice Sylvestre

August 27, 2003

A. Scope: Calibration of Hydrologic Models. Calibration of dynamic wave
channel routing models is covered in another document.

B. Hydrologic Model Calibration

[B.1. Upgraded Calibration System]{.underline}

Perhaps the most effective changes can be achieved by upgrading the
Calibration System (Smith et al. 2003). As an example of the dramatic
increases in process efficiency that can be realized, several years ago
Heather Friedeman, JJ Pan, and Mike Smith developed the Interactive
Double Mass Analysis (IDMA). This was a GUI tool that replaced the
manually intensive graphical procedure for performing a data quality
control procedure called \>double mass analysis=. This procedure is a
standard and widely accepted step in hydrologic data analysis and is
performed on gage observations of rainfall, temperature, potential
evaporation, and streamflow (Linsely et al. 1982). (It is analogous to
the need for an MPE Re-analysis of the historical Nexrad radar
products). After this tool was delivered to the field, Geoff Bonnin
commented that one or more RFCs reported that this part of the quality
control process for model calibration was reduced from \>months to
weeks=. While IDMA was somewhat of a crude combination of existing
programs, it nonetheless provided dramatic improvements.

Eric Anderson has completed the definition of the Functional
Requirements for an enhanced Calibration System. Such a system would
more effectively tie together all the \>legacy= programs and procedures
that were developed on the NAS 9000 and simply ported to the HP
workstations. New tools would also be added, such as the data mining
tools developed at the University of Arizona and U. of California at
Berkeley. The new system would combine the functions and data now
present in:

Calibration Assistance Program (CAP)

Interactive Calibration Program (ICP)

Interactive Double Mass Analysis Program (IDMA)

Calibration programs MAP, MAT, PXPP, TAPLOT, MAPE, and MAPX

Archives of RFC data (critical for FloodWave Calibration)

New versions of Data (e.g., potentially John Schaake=s data)

Automatic calibration procedures.

As new models are developed, it is critical that a commensurate
calibration system is developed for field implementation.

[B.2. Continued Research]{.underline}

This item primarily addresses the need for continued long term research
and development into automatic optimization methods for model parameter
calibration. It is the opinion of Victor Koren, Eric Anderson, Mike
Smith, and the University of Arizona that the current single

objective optimization procedures developed to date cannot duplicate the
quality of manual calibration (Schaake, 2003). However, there is great
potential in multi-objective automatic optimization and its derivatives
to provide efficiency in the calibration process while achieving the
quality and spatial consistency achieved through manual calibration.
This is a very active area of research, and the University of Arizona is
perhaps the leader in this area. Also, Vadim Kuzmin is looking at the
automatic calibration process from a new perspective. In addition, he is
developing a procedure to evaluate the goodness of the parameters
derived by automatic calibration (his so-called F-Robustness approach).

Research is also needed on the derivation of initial (*a priori*) values
of model parameters, both for lumped and distributed models. Victor
Koren pioneered the derivation of SAC-SMA parameters from NRCS course
resolution soils information and delivered these to the RFCs via CAP.
Experience with these parameters has shown the need for further work.
Richard Anderson has adapted Victor=s theory to derive SAC-SMA
parameters from high resolution NRCS soils information. In some cases
simulation improvements have been seen compared to those derived from
coarse resolution soils data.

[B.3. RFC Implementation]{.underline}

Efficient model calibration is dependent on the RFC organization as
well. Smith et al. (2003) described the well-organized approach at the
NWRFC. Such well-organized approaches have led to the completion of
model calibration at NWRFC, CBRFC, and CNRFC, with the additional
benefit that forecasters are better trained (Don Laurine, personal
communication). Mike Smith and Eric Anderson have advocated such
organized approaches at every calibration workshop for the last 10
years. However, it is up to the individual RFC leadership to stress the
importance of such an approach.

[References]{.underline}

Linsley, R.K., Kohler, M.A., and Paulhus, J.L.H., 1982. Hydrology for
Engineers. McGraw-Hill, New York.

Smith, M.B., Laurine, D.P., Koren, V.I., Reed, S.M., and Zhang, Z.,
2003. Hydrologic Model Calibration in the National Weather Service. In:
\>Calibration of Watershed Models=, Water Science and Applications 6,
Qingyun Duan, Hoshin Gupta, Soroosh Sorooshian, Alain Rousseau, Richard
Turcotte, Editors. American Geophysical Union, 133-152.

Schaake, J. C., 2003. Introduction. In: \>Calibration of Watershed
Models=, Water Science and Applications 6, Qingyun Duan, Hoshin Gupta,
Soroosh Sorooshian, Alain Rousseau, Richard Turcotte, Editors. American
Geophysical Union, 1-7.

Appendix 3

Calibration Assistance Program

Summary of Capabilities and Data Sets

<http://www.nws.noaa.gov/oh/hrl/calb/esrihtmlfinal/p1082.htm>

A. Capabilities

The CAP is an Arc/View application. Its capabilities include the
following.

1\. Derive area-elevation curves

2\. Sub-divide basins based on elevation zones

3\. Derive elevation-precipitation plots

4\. Display defined zones on top of other data layers (e.g.
precipitation, elevation)

5\. Compute basin mean, max, and min values of: (may also compute for
each elevation zone defined within a basin)

5.1 precipitation (monthly, annual, and seasonal)

> 5.2 potential evaporation (monthly, annual, and seasonal)

5.3 potential evaporation adjustment factors

5.4 percent forest

5.5 percent of each forest type

> 5.6 soil-based estimates for 11 SAC-SMA parameters

6\. Display selected NOHRSC historical snow images from (1990-1995)

2\. Data Sets

1\. Digital Elevation Model (DEM) 4km resolution

2\. Percent of forest cover on 1km grid

3\. Forest type on 1km grid

4\. PRISM annual and monthly precipitation grids.

5\. Annual and monthly potential evaporation grids

6\. Soil type, depth, and texture for 11 layers

7\. Snow cover grids for 1990-1995

8\. State boundaries

9\. EPA River Reach Files (RF1)

10\. SAC-SMA parameter grids (1km x 1km)