________________________________

Corresponding author address: Cody L. Moser, Ph.D., National Weather Service, 328 Innovation Blvd., State 

College, PA 16803. E-mail: cody.lee.moser@noaa.gov

Eastern Region Technical Attachment

No. 2013-01

June 2013

Comparison of the SAC-SMA and API-CONT Hydrologic Models at Several Susquehanna 

River Headwater Basins

Cody L. Moser, Scott Kroczynski, and Kevin Hlywiak

NOAA/NWS/Middle Atlantic River Forecast Center

State College, Pennsylvania

ABSTRACT

Hydrologic simulation comparisons were made between the Sacramento Soil Moisture 

Accounting (SAC-SMA) model and the Continuous Antecedent Precipitation Index (API-CONT) 

model for six Susquehanna River Basin (SRB) headwater basins. Using a 6-hour time step, the 

difference in cumulative simulation error (stage) between observations and model simulations

was calculated over a 19-month period (August 2010-February 2012) to visualize event, 

monthly, and seasonal model trends. Next, simulated crests (stage) were compared to observed 

crests for five rain-driven events. Finally, Ensemble Streamflow Prediction (ESP) was used on a 

daily time scale to provide a long-term (1950-1998) perspective of model performance compared 

to historical high flow events. Results show that the SAC-SMA model consistently produced more 

accurate streamflow simulations in terms of the cumulative error in stage over time. However, 

we could not establish a distinct seasonality (i.e. wet or dry periods, snowmelt) pattern in which 

one model was consistently more accurate across all basins. The API-CONT model more 

accurately predicted large event (flood) crests based on the Probability of Detection (POD), 

False Alarm Rate (FAR), Critical Success Index (CSI), and bias verification statistics. Results 

suggest that use of the lumped SAC-SMA could improve upon the lumped API-CONT for low and 

medium flow forecasts, which may be particularly useful for longer lead time applications such 

as water supply management. However, no improvements would be expected for flood forecasts.

_______________________________________
2

1. Introduction

Thirteen regional River Forecast Centers 

(RFCs) within the National Weather Service 

(NWS) provide a suite of streamflow 

forecast products. The forecasts predict 

future stream height and flow and extend 

from hours (short-term) to months 

(long-term) into the future. The forecasts aid

the general public, community leaders, 

emergency managers, and 

reservoir/hydropower managers in making 

better life- and cost-saving decisions.

Two primary hydrologic (i.e. rainfall-runoff)

models, the Sacramento Soil Moisture 

Accounting (SAC-SMA) model and the 

Continuous Antecedent Precipitation Index 

(API-CONT) model, are used within RFCs 

to produce daily streamflow forecasts. 

Currently, most implementations of these

models are lumped (not distributed) and run 

operationally on a 6-hour time step. A 

lumped modeling approach assumes 

uniformly distributed precipitation and/or 

melt in both time and space across defined 

watersheds. Distributed hydrologic models 

involving non-homogenous watershed 

characteristics are currently used for 

developing flash flood guidance but are not 

widely used for point forecasting on larger 

rivers. RFCs commonly use hydrologic 

routing techniques to route flows from 

headwater sub-basins to downstream points. 

Where necessary to account for dynamic 

hydraulic effects, RFCs also use the U.S. 

Army Corps of Engineers (USACE)

Hydrologic Engineering Centers River 

Analysis System (HEC-RAS) unsteady flow 

model. During the cold season, a snow

accumulation and ablation model 

(SNOW-17), described by Anderson and 

Crawford (1964), Anderson (1968), and 

Anderson (1973), is coupled with the 

API-CONT and SAC-SMA models to 

represent and compute physical snowpack 

processes (i.e. snow water equivalence, 

snowpack accumulation and melt).

The NWS Middle Atlantic River Forecast 

Center (MARFC) is currently the only RFC 

to use the API-CONT model for streamflow 

prediction. The API model was originally 

developed by Linsley et al. (1949) to be

applied on an event by event basis. In API 

models, soil moisture gain is tracked by an 

Antecedent Precipitation Index, which is 

adjusted for seasonal variations in 

evaporation and transpiration to produce an 

estimate of runoff for a given rainfall/melt. 

The API model consists of empirical 

relationships derived from historical 

precipitation, potential evapo-transpiration

(PE), and streamflow data, and generates 

storm runoff based on antecedent moisture 

conditions, a seasonality function, storm 

duration and total precipitation (Smith et al. 

2000). A continuous API-type model was 

developed by Sittner et al. (1969) in order to 

accommodate the need for longer-term 

water-management forecasts and eliminate 

some of the calibration challenges involved

with an event-based API model. The 

MARFC API-CONT model is an adaptation 

of the Sittner et al. (1969) model (NWS, 

2013). The API-CONT model computes 

runoff on an incremental (continuous), not a 

storm, basis and generates both surface and 

baseflow runoff amounts.

The SAC-SMA model was initially 

developed by Burnash et al. (1973) and is 

used operationally by the remaining 12 

RFCs. The model is conceptual and 

represents the soil profile as a system of two 

layers (zones), each having a tension water 

and one or more free water components or 

reservoirs (Smith et al. 2000). Applied 

moisture is distributed in a physically 

realistic manner within the various zones 

and energy states in the soil, allowing for the 
3

preservation of rational percolation 

characteristics.

The API-CONT and SAC-SMA hydrologic 

models can be used in conjunction with 

user-specified modifications (MODS) to 

account for non-standard conditions and to 

keep the model states on track (Smith et al. 

2000). Warm season model modifications 

common between both models include the 

ability to change precipitation amount, 

runoff volume, and baseflow. Common 

modifications when using SNOW-17 with 

either of the two rainfall-runoff models

include changing precipitation type (rain or 

snow), snowmelt rate, snow covered area,

and snow water equivalent amount.

The MARFC is considering switching from 

the API-CONT model to the SAC-SMA 

model. Incorporating the SAC-SMA model

into MARFC forecasting offers several

potential advantages. These include: (1) 

better RFC homogeneity regionally and 

nationally, (2) increased NWS Office of 

Hydrologic Development (OHD) and NWS 

hydrologic contractor support and, (3) 

receiving, implementing, using new 

science/technology faster. Costs and 

calibration time required to implement the 

SAC-SMA are potential disadvantages that 

must also be considered. Based on the 

Northeast River Forecast Center’s (NERFC) 

experience in switching to the SAC-SMA 

model, a full MARFC transition would 

likely take at least five years and cost over 

one million dollars in RFC personnel time or 

contractor costs (Rob Shedd, NERFC). A 

better investment may be exploring 

improved modeling approaches including 

distributed models. This study does not 

attempt to weigh these considerations. 

Rather, the purpose of this study is to help 

inform MARFC’s future modeling decisions 

via a comparison evaluating the operational 

performance advantages and disadvantages

of both hydrologic forecast models in the 

Mid-Atlantic region.

More specifically, two primary goals were 

established at the beginning of this project. 

First, we wanted to compare the difference 

in effort needed for maintaining the model 

states between the two models on an 

operational (i.e. daily) basis. The second 

goal was to compare the simulation accuracy 

of the two models. In this region, flood 

events dominate the hydrologic forecast 

focus, not typically drought or low flow 

conditions, as in other regions of the country

where water supply management issues are 

more common. For this reason, more 

attention was given to model simulations 

during high flow events in this study, and 

less consideration was given to low and 

medium flow periods.

Six MARFC headwater forecast points in 

the Susquehanna River Basin (SRB) were 

used in this study (Fig. 1, Table 1). Each of 

the six MARFC forecast points is 

represented by an operating United States 

Geological Survey (USGS) stream gauge

which defines the outlet of each basin. 

Hydrologic model (API-CONT/SAC-SMA) 

basin parameters used in this study were 

calibrated using observed stage/flow data 

from the USGS stream gauge located at the 

outlet of each basin. Model performance in 

this study was evaluated by comparing each 

model simulation with observed flow/stage

at the USGS stream gauge location.

The SRB has a non-tidal influenced drainage 

area of approximately 27,000 square miles 

and covers parts of the states of New York, 

Pennsylvania, and Maryland. Six major 

sub-basins comprise the SRB: North Branch, 

Chemung, Upper Main Stem, West Branch, 

Juniata, and Lower Main Stem. The 

Susquehanna River is the longest, 

commercially non-navigable waterway in 
4

the U.S., and is the largest tributary to the 

Chesapeake Bay, supplying roughly 50% of 

its freshwater inflow (SRBC, 2012). The

combination of climate, terrain, and 

floodplain development make the SRB one 

of the Nation’s most flood-prone areas.

Widespread floods are most common during 

the spring snowmelt and tropical seasons, 

but flooding can occur anytime during the 

year (Fig. 2).

Note that “flooding” at a given location 

depends not only on the hydroclimatology 

and hydrology of the basin, but also on the 

NWS flood stage at the outlet which is a 

function of flood impacts on humans. The 

SRB has an average annual precipitation of 

approximately 40 inches. All six headwater 

points in this study have a drainage area less 

than 500 square miles and normally crest

within 24 hours after a rainfall event begins.

Figure 1. Map showing SRB, study basins, and USGS gauges included in the study.

Table 1. SRB NWS daily streamflow forecast points and sub-basins included in the study.

NWS Forecast 

Point

Susquehanna 

Sub-Basin Stream State

Crest 

Time (hr)

Area 

(mi2)

Gauge 

Elevation (ft)

NWS 

ID USGS ID

Campbell Chemung Cohocton River NY 18 470 1,016 CMPN6 01529500

Cortland North Branch Tioughnioga River NY 24 292 1,085 CRTN6 01509000

Harper Tavern Lower Main Stem Swatara Creek PA 18 337 357 HTVP1 01573000

Shirleysburg Juniata Aughwick Creek PA 24 301 570 SLYP1 01564512

Spruce Creek Juniata Little Juniata River PA 12 220 751 SPKP1 01558000

Tunkhannock Upper Main Stem Tunkhannock Creek PA 12 383 610 TNKP1 01534000
5

Figure 2. Historical (a) flow regime and (b) flood regime of the basins included in the study. 

Note that Fig. 2a excludes stream gauge SLYP1 due to its short period of record.

2. Data and Methodology

To gain an operational perspective on the 

similarities and differences between model 

components, a senior hydrologist familiar 

with the API-CONT model applied 

user-specified MODS to both models. 

MODS were made on an event basis and 

only if they were considered necessary by 

the hydrologist. The differences, challenges, 

and knowledge required to maintain model 

states for each of the two models were 

noted. The hydrologist had limited 

knowledge of the SAC-SMA model 

parameters and characteristics. This part of 

the study was used only to acquire a basic

understanding of model response to 

user-specified MODS and to compare the 

effort required to similarly maintain the 

states of both models. However, the 

“modified simulations,” although generated, 

were not used in this study to quantify the 

hydrologic performance of the two models. 

Rather, only “pure simulations” (i.e., no 

MODS) were used, thereby eliminating any 

potential forecaster bias.
6

For quantitative analysis, three streamflow 

data types were used - USGS stream gauges 

provided observed data (streamflow and 

stage; http://www.usgs.gov/water/), while 

simulated runoff from the two models 

(API-CONT and SAC-SMA) were used as 

predicted data. Using the basin unit 

hydrograph and gauge rating curve, 

simulated runoff is converted to a simulated 

flow and stage at the gauge location. The 

simulation performance of the two 

hydrologic models was assessed using three 

different methods over varying time scales. 

Each of the three methods is discussed in 

more detail in following paragraphs. In all 

analyses, model simulations used identical 

observed mean areal precipitation and mean 

areal temperature (MAT) forcings. Prior to 

the actual analysis period (August 2010

through February 2012), both models were 

run for a minimum of three months to 

initialize the model states. Model 

simulations were made using radar-based 

(derived) Mean Areal Precipitation (MAPX) 

on a six-hour time scale. MAPX data are 

created using both radar estimated

precipitation and gauge observations. 

MARFC started using MAPX as the default 

input precipitation for all daily forecast

points in November 2008. Prior to then, 

MARFC used gauge-based (derived) Mean 

Areal Precipitation (MAP) on a six-hour 

time scale as the default input precipitation. 

Although sometimes forecasters will choose 

to use MAP data in operations because there 

can be differences between MAPX and 

MAP that vary seasonally, we chose to use 

the MAPX data for the 2010-2012 period in 

this study because it is the best available for 

most of the period. For the period 1950 

through 1997, “historical” simulations were 

made using Mean Areal Precipitation 

(MAP) since MAPX data were not 

available. MAP data uses only precipitation 

gauge observations and does not include

radar-estimated precipitation. Note that both 

the API-CONT and SAC-SMA models were 

calibrated using MAP data, so any 

differences in calibration and validation 

forcings will be the same for both models. 

For reference, both Cognitore (2005) and 

Zhang et al. (2010) report on differences 

between MAP and MAPX in the MARFC 

data archives.

Important API-CONT and SAC-SMA 

calibration differences exist in our study. 

First, the API-CONT model was calibrated 

for each of the six sub-basins by several 

MARFC hydrologists over time. 

Furthermore, the MARFC’s primary goal is 

to most accurately forecast large event

(flood) crests. Because of this, the calibrated 

model parameters tend to be skewed toward 

high flow events and are consequently less 

effective in lower flow events. Second,

Riverside Technology, inc. (RTi) was tasked 

with calibrating the SAC-SMA model for 

use in this study (RTi, 2009). Unlike 

MARFC’s calibration approach, RTI’s 

calibration procedure focused on generating 

the best overall calibration statistics and was 

not skewed toward high flow events. RTi’s

calibration process included developing new 

unit hydrographs more compatible with the 

SAC-SMA model’s treatment of surface vs. 

baseflow runoff. In picking events, RTi 

generally used the following criteria in 

developing unit hydrographs during their 

calibration process: (1) an event should be 

isolated from other events and ideally, there 

should be several dry days prior to and after 

the precipitation event, resulting in a smooth 

and continuous hydrograph with minimal 

interference from other events, (2) an event 

should be free from obvious measurement 

noise, (3) ‘medium-sized’ events were 

preferred for analysis, (4) multiple-peaking 

events should not be used because they are 

indicative of non-constant runoff rates, and 

(5) events that are influenced by ice storms 

or snowmelt should not be used. RTI’s
7

evaluation of SNOW-17 and SAC-SMA 

model parameters was based on the visual 

closeness of individual observed and 

simulated hydrographs as well as overall 

simulation error statistics. During RTi’s 

calibration phase, multiple calibrators 

reviewed parameters and statistics and 

worked together to attain appropriate 

parameter sets. RTi calibrated five of the six 

sub-basins included in this study (Cortland, 

Harper Tavern, Shirleysburg, Spruce Creek, 

and Tunkhannock). The SAC-SMA 

calibration for Campbell was performed by 

two MARFC hydrologists during a brief

familiarization exercise and is therefore 

considered incomplete, having not 

undergone the same rigorous calibration 

techniques used by RTi. However, Campbell 

SAC-SMA simulations were still included in 

this study for comparison purposes.

The first quantitative method that compared

the two model simulations used a

cumulative simulation error approach. For 

each of the six streamflow gauges, simulated 

stream height (stage) was compared to 

observed stage using a 6-hour time step 

from August 2010 to February 2012 (19 

months), and the respective absolute errors 

were computed and accumulated over the 

study period. The benefit to using this 

approach is that it provides visual insight on 

model behavior on multiple time scales (i.e. 

events, wet/dry periods, and seasons). Next, 

the difference in cumulative simulation error 

values between models was calculated at 

each time step, providing an error difference

time-series. The slope of the difference

time-series is significant because based on 

whether the slope of the time-series line is 

positive or negative indicates which model 

performed better during a particular event, 

wet/dry period, or season.

The second method made crest comparisons

at each streamflow gauge. First, simulated 

crests (stage and flow) for each model were

compared with observed crests for five 

events from August 2010 to February 2012 

(19 months). Events were chosen based on 

magnitude and precipitation type. The five 

largest events in which rain was the sole

precipitation type (with no snow melt 

effects) were evaluated at each of the six 

streamflow gauges independently. 

Instantaneous (15-minute) observed crest 

data was compared with 6-hour simulated 

crest data. Next, a contingency table was 

used to describe the distribution of 

simulations and observations in terms of 

categorical (i.e. flood vs. no flood)

frequency (Table 2). Only flood (observed 

and/or simulated) occurrences within the 

five chosen events at each gauge were 

considered for the contingency table. The 

calibration period for each model did not 

include the five events used for crest 

comparisons in this study. Therefore, 

evaluating each hydrologic model 

simulation during the five high flow events 

provided independent verification 

information.

For the crest analysis comparison, numerous 

verification statistics were calculated and 

expressed in percentages (multiplied by 

100). Probability of Detection (POD or hit 

rate) is the proportion of observed floods 

that were simulated to be floods. POD 

values (scores) range from 0 (worst) to 100

(perfect). False Alarm Rate (FAR) is the 

proportion of simulated floods that were not 

observed to be floods, and values range from 

0 (perfect) to 100 (worst). Critical Success 

Index (CSI or threat score) is the proportion 

of correctly simulated floods over all floods, 

either simulated or observed. CSI values 

range from 0 (worst) to 100 (perfect). Bias is 

the ratio of all simulated floods over all 

observed floods. Bias scores range from 0 

(low bias) to ∞ (high bias), with a value of 

100 being perfect. Model simulation 
8

verification scores provided valuable 

comparison statistics for numerous flood 

events across multiple gauges.

POD = a / (a+c) (1)

FAR = b / (a+b) (2)

CSI = a / (a+b+c) (3)

Bias = (a+b) / (a+c) (4)

Where a, b, and c are defined in the 

contingency table (Table 2).

Table 2. Contingency table used to calculate 

the simulated verification scores.

Event 

Observed

Yes No Total

Event 

Simulated

Yes a b a + b

No c d c + d

Total a + c b + d

a + b

+ c +

d = n

A two-sample t-test was used for the crest 

comparison analysis to determine the 

probability (p-value) that the simulated crest 

error from one model was significantly 

different from the simulated crest error of 

the other model. For the test, a t-value is 

calculated and compared with a standard 

table of t-values to determine whether the 

t-statistic reaches a certain threshold of 

statistical significance. The two-sample 

t-test was performed including the Campbell 

gauge (n = 30 events) and excluding the 

Campbell gauge (n = 25 events). The 

absolute value of the simulation error was 

used for each model during all events.

t = ( 1 - 2) / √(s1

2

/n1 +s2

2

/n2) (5)

where:

= the average model simulation error for 

all events

s = the standard deviation of the model 

simulation error for all events

n = total number of events

Finally, for the third method of quantitative 

comparison, the NWS Extended Streamflow 

Prediction Analysis and Display Program

(ESPADP) was used to compare the 

hydrologic model simulations on a historic 

(>30 years) time frame. ESPADP was 

originally developed to produce probabilistic 

long lead-time conditional hydrologic model 

simulations over a user-specified time 

period for water supply forecasting. 

ESPADP also has the capability to generate 

daily historic model simulations for each 

year of available record using historical 

observed forcings (MAP/MAT/PE/snow 

melt). For this study, ESPADP was used to 

compare model exceedance probability 

curves with the observed exceedance 

probability curve at each gauge. For our 

application, we used the ESPADP option to 

select the maximum daily flow value in each 

historical year and then used those values to 

create an exceedance probability plot. We 

plotted these exceedance curves for both 

simulated and observed data. This method is 

an efficient way to compare simulated and 

observed flows for numerous (~50) high 

flow events across many years. Since 

ESPADP simulations are based on average 

daily flow, the crest (peak flow) is not 

captured, which may lead to different results 

than the crest analysis comparison.

3. Results

a) Maintaining Model States

As stated earlier, a senior hydrologist 

familiar with the API-CONT model applied 
9

user-specified modifications (MODS) to 

both models on an event basis but only if 

they were considered necessary. The 

hydrologist who generated the MODS noted 

that in general the effort required to 

maintain model states within each of the two 

models was similar. Since the hydrologist 

had only limited knowledge of the 

SAC-SMA model parameters and 

characteristics, there was a learning curve 

associated with making MODS using the 

SAC-SMA model as well as determining 

what the effects of those MODS were. 

However, once a comfort level was 

achieved, the actual effort in keeping the 

model states similar for the two models was 

comparable. More active 

hydrometeorological periods naturally 

require more maintenance (i.e., more 

MODS) in each model, while during less 

active periods the hydrologist found that the 

SAC-SMA seemed to require less 

maintenance than the API-CONT model. In 

summary, the hydrologist noted no 

significant difference in terms of time, effort 

or degree of complexity related to 

maintaining the model states of each of the 

two models.

b) Simulation Performance - Cumulative 

Simulation Error and Error Difference Time 

Series

Cumulative simulation error results can be 

interpreted as follows. First, colored lines 

(API-CONT=blue, SAC-SMA=red) 

representing modeled cumulative simulation 

error were plotted over the observed (black)

time series (Figs. 3a-8a). The line with the 

smallest cumulative simulation error at the 

end of the time series is the more accurate 

model over the 19-month period (August 

2010 through February 2012). Next, the

difference in simulation error (red line 

minus blue line) between both models was

calculated and provided an error difference 

time series (green line; Figs. 3b-8b). The 

green line has a positive slope when the 

API-CONT is more accurate and a negative 

slope when the SAC-SMA model is more 

accurate. The steepness of the error 

difference time series should also be 

considered. The models are in general 

agreement when the error difference 

time-series is horizontal. A steep, positive 

slope indicates a period when the 

API-CONT model outperformed the 

SAC-SMA model. A steep, negative slope 

indicates a period when the SAC-SMA

model outperformed the API-CONT model.

A bump in the error difference time series 

indicates an event in which the API-CONT 

was more accurate while a dip indicates an 

event when the SAC-SMA was more 

accurate. The cumulative simulation error 

and error difference time series are plotted 

with the observed time series to provide 

visual analysis of daily model performance

for the length of study period.

Overall, the SAC-SMA model was more 

accurate over the 19-month period at five of 

the six gauges using the cumulative 

simulation error method. Campbell was the 

only location where the API-CONT model

outperformed the SAC-SMA model, and this 

was likely due to the lower quality 

SAC-SMA calibration process mentioned 

earlier. However, we could not determine a 

consistent pattern in which one model 

outperformed the other at all gauges during 

a particular season. Instead, results suggest 

that both models simulate streamflow 

similarly the majority of the time (i.e. 

horizontal or relatively flat slope error 

difference time-series). Our results compare 

favorably with RTi’s calibration report (RTi, 

2009) – that in most cases, the SAC-SMA 

model is able to simulate the cumulative 

water balance more accurately than the 

API-CONT model.
10

Figure 3. Campbell simulation results. (a) Cumulative simulation error: the API-CONT model 

had lower simulation error over time; (b) Error difference time series: the models were in general 

agreement for most of the study period. The API-CONT model outperformed the SAC-SMA 

model for the July-October months.
11

Figure 4. Cortland simulation results. (a) Cumulative simulation error: the SAC-SMA model had 

lower simulation error over time; (b) Error difference time series: the models were in general 

agreement for most of the study period. The SAC-SMA model outperformed the API-CONT 

model from September 2011 through December 2011.
12

Figure 5. Harper Tavern simulation results. (a) Cumulative simulation error: the SAC-SMA 

model had slightly lower simulation error over time; (b) Error difference time series: model 

seasonality patterns were unclear.
13

Figure 6. Shirleysburg simulation results. (a) Cumulative simulation error: the SAC-SMA model 

had lower simulation error over time; (b) Error difference time series: the models were in general 

agreement for most of the study period. The SAC-SMA model outperformed the API-CONT 

model for the late summer/early fall months.
14

Figure 7. Spruce Creek simulation results (a) Cumulative simulation error: the SAC-SMA model 

had lower simulation error over time; (b) Error difference time series: the models were in general 

agreement for most of the study period. Model seasonality patterns were unclear.
15

Figure 8. Tunkhannock simulation results: (a) Cumulative simulation error: the SAC-SMA 

model had slightly lower simulation error over time; (b) Error difference time series: the 

SAC-SMA model outperformed the API-CONT model during winter months 

(November-February). The API-CONT model outperformed the SAC-SMA model during 

snowmelt and early summer months (March-June).
16

c) Simulation Performance - Crest Analysis

The five largest rain-dominant events were 

chosen for each basin (independently) from 

August 2010 to February 2012. All chosen 

events were common for at least two of the 

basins and included: Tropical Storm Nicole 

(October 2010), an early December 2010 

rain event, a cold heavy rain event (March 

2011), an early summer rain event (April 

2011), Tropical storm Lee (early September 

2011), a non-tropical event (late September 

2011), and a late fall event (November 

2011). The Campbell gauge did not 

experience any observed flooding during the 

19-month period. However, at the other five

gauges, a minimum of two of the five 

chosen events were flood events and many 

times at least four of the five chosen events 

were flood events. As an example, the 

procedure for choosing the events and 

computing the verification statistics are 

shown in Figure 9 at the Harper Tavern 

gauge. The average absolute simulated crest 

error for each model was computed for all 

events at each gauge and is provided in

Table 3.

In summary, the API-CONT model had a 

lower average simulated error at all gauges 

(Table 3). In Figure 10, the ‘crest’ 

verification statistic was based on all chosen 

events and indicated the percent of time that 

each model simulated the crest more

accurately. The POD, FAR, CSI, and Bias 

statistics required either observed or 

simulated flooding to occur, which 

represented 25 of the 30 events when the 

Campbell gauge was included (Fig. 10a) and 

22 of 25 events when the Campbell gauge

was excluded (Fig. 10b). When including 

the Campbell gauge in the analysis, the 

API-CONT model outperformed the 

SAC-SMA model for all of the event 

verification statistics. Excluding the 

Campbell gauge in the analysis resulted in 

improved SAC-SMA verification statistics, 

but the API-CONT model still outperformed 

the SAC-SMA model for all of the event 

verification statistics except FAR. 

Furthermore, the two-sample t-test indicates 

that the difference in API-CONT and 

SAC-SMA simulation errors is statistically 

significant. Including the Campbell gauge

resulted in a p-value < 0.02, meaning there 

is a 98% probability that the API-CONT 

outperformed the SAC-SMA model during 

the 30 selected crest events. Excluding the 

Campbell gauge resulted in a p-value < 0.05, 

meaning there is a 95% probability that the 

API-CONT outperformed the SAC-SMA 

model during the 25 crest events. The 

API-CONT model most likely outperforms 

the SAC-SMA model during high flow 

events due to the previously mentioned 

difference in calibration approaches. Our 

results agree with RTi’s calibration report -

that MARFC’s current API model often 

provides similar or better simulations of 

large precipitation events.
17

Figure 9. Choosing the five largest rain driven events at the Harper Tavern gauge. (a) Five 

chosen events which were all flood events. (b) Crest comparison results (Observed = black, 

API-CONT = blue, SAC-SMA = red). A similar procedure was used at all gauges.

Table 3: Average absolute crest error (ft) for the 5 chosen events at each gauge.

Location API-CONT SAC-SMA

Campbell 0.57 1.60

Cortland 0.98 2.09

Harper Tavern 2.71 3.95

Shirleysburg 1.46 3.08

Spruce Creek 1.36 1.63

Tunkhannock 1.59 1.87
18

Figure 10. Event verification statistics (a) including Campbell and (b) excluding Campbell.

d) Simulation Performance - Extended 

Streamflow Prediction (ESP) Analysis

The historical data period in ESP varied at 

each gauge. Campbell, Cortland, Spruce 

Creek, and Tunkhannock ESP analysis 

included the period from 1950 to 1997 (48 

years). The period from 1950 to 1988 (39 

years) was used at Harper Tavern. 

Shirleysburg was not included in the ESP 

analysis due to limited historical gauge data.

Three exceedance probability curves 

(observed, API-CONT, SAC-SMA) were 

created for each gauge. The curves were 

based on annual maximum daily flow 

values. Increased flow corresponds to 

decreased exceedance probability. For 

example, if 50 years of historical data are 

available, the 2% exceedance probability 

flow would correspond to the year (trace) 

with the largest observed maximum daily 

flow. Since ESP includes one value per year 

in the output, the 98% exceedance 

probability value would correspond to the 

year (trace) that included the 50th largest

maximum daily flow value.

The API-CONT model simulated the 

historic flow regime more accurately at 

Campbell, Harper Tavern, and Spruce Creek

(Fig. 11). The SAC-SMA model simulated 

the historic flow regime more accurately at 

Cortland and Tunkhannock. The API-CONT 

model simulated the 2% exceedance flow 

more accurately at every gauge. ESP 

analysis confirms that both models 

simulated the upper end of the historical 

daily flow regime at each gauge with similar 

trends.
19

Figure 11. ESP results for (a) Campbell, (b) Cortland, (c) Harper Tavern, (d) Spruce Creek and, 

(e) Tunkhannock. Plots show the largest (one per year) simulated or observed values.
20

4. Conclusions

We performed a comprehensive comparison 

study between the API-CONT and 

SAC-SMA hydrologic models at six

headwater basins in the SRB. The 

SAC-SMA model simulated streamflow 

more accurately on a daily basis over a 

19-month period (August 2010 through 

February 2012) using a cumulative 

simulation error and error difference time 

series method of analysis. The API-CONT 

model simulated large event crests more 

accurately as shown by verification statistics 

including POD, FAR, CSI, and Bias. 

Finally, using ESP, both models simulated 

the upper end of the historical daily flow 

regime for each basin similarly. The results 

are consistent with the calibration goals that 

were established for each of the models. 

Unfortunately, the results would be more 

meaningful if both models were calibrated 

by a single entity and followed a similar 

calibration process, but that preference was 

not possible. If desired, MARFC could 

likely make adjustments to the SAC-SMA 

calibration parameters to better simulate 

high flows while limiting the simulation 

performance at low to medium flows. With 

respect to the effort required to reasonably 

maintain the operational model states of the 

two models by using forecaster-generated 

MODS, we found no important difference. 

Naturally, there is a learning curve 

associated with using the SAC-SMA model 

(and in making and understanding the 

MODS) for hydrologists and RFCs that have 

never used the model (i.e., MARFC).

Smith et al. (2000) performed a similar 

study at a total of 3 stream gauges (two in 

Iowa and one in Georgia). They evaluated 

the advantages of the continuous SAC-SMA 

model over an event API model. Using 

hydrograph shape error and peak error, they 

found that the SAC-SMA model simulated 

flow more accurately across all of the 

gauges for a majority of chosen flow 

intervals. Compared with Smith et al. 

(2000), our results suggest that a continuous 

API model simulates peak flows comparable 

to the SAC-SMA model. One reason our 

results differ may be due to using 

continuous API model simulations instead 

of event based API model simulations. 

Another reason may be due to significant 

differences in basin size, topography, and 

rainfall-runoff response time.

Predictions of water across the United States 

are important to making decision on water 

management, recreation, and natural 

hazards. RFCs have different forecast 

verification goals based on regional water 

needs and concerns. In the Western U.S., 

long-term water supply forecasting plays an 

important role in RFC operations, and the 

SAC-SMA model is more suited for the 

region. Eastern U.S. RFCs, such as 

MARFC, are currently more concerned with 

flood crest forecasting and water supply 

forecasting plays less of a role. Our analyses 

indicate that the MARFC calibrated 

API-CONT model outperformed the RTi 

calibrated SAC-SMA model when 

forecasting large event crests. This agrees 

with MARFC forecaster experience and 

flood event reviews that indicate the APICONT model has continued to be an 

effective tool for MARFC flood forecasting.

During the last few years, major advances 

have been accomplished within the

NOAA/NWS operational hydrologic 

forecasting framework. The implementation 

of a new software modeling and operational 

infrastructure known as the Community 

Hydrologic Prediction Service (CHPS) is 

expected to be the foundation which will 

help NOAA/NWS meet goals for hydrologic 

technology transfer and improved 

hydrologic operations, services, and 
21

products in the future (Roe et al. 2010). 

Essentially, CHPS is an open, modular 

hydrologic forecasting system that allows 

existing and new hydraulic and hydrologic 

models and data to be utilized and ultimately 

shared by members of the hydrologic 

community.

Within NOAA/NWS, CHPS brings a new 

flexibility to operational hydrology. With 

proper implementation and configuration, 

CHPS should allow for the real-time 

operational use of multiple hydrologic 

models. In the context of this paper, CHPS 

should allow for both the API-CONT and 

SAC-SMA hydrologic models to be run 

concurrently so that comparisons of the 

output (i.e., hydrologic forecasts) of the two 

models can be made in a near real-time

operational environment. This “ensemble” 

approach would be analogous to that being 

utilized in the NOAA/NWS operational 

meteorology environment, where viewing 

and interpreting the output from multiple 

atmospheric models concurrently in near 

real-time is already being done. As such, 

possible future work to build upon this paper 

might include MARFC adding, configuring 

and running the SAC-SMA in CHPS for 

these same five basins for use in daily 

operations. The running of two distinct 

hydrologic models within CHPS would be a 

valuable test of the presumed flexibility that 

the system should provide and would also 

allow for direct comparison of model 

performance over all flow ranges and in a 

real-time operational environment. An 

added benefit would be MARFC staff 

exposure to (and experience gained in using) 

the SAC-SMA model. Depending on the 

results and consideration of the advantages 

and disadvantages mentioned in the 

introduction, MARFC could then consider 

expanding the number of calibrated 

SAC-SMA basins for future operational use.

ACKNOWLEDGEMENTS

The authors thank Seann Reed, 

Development and Operations Hydrologist at 

the MARFC, and Peter Ahnert, Hydrologist 

in Charge (HIC) at the MARFC, for their 

review of this manuscript. The authors also 

thank Ned Pryor, Senior Hydrologist at the 

MARFC, for assisting us in setting up and 

running the ESPADP software for this 

study. Lastly, the authors want to thank Rob 

Shedd (HIC, NERFC) and Pedro Restrepo 

(HIC, NCRFC) for their reviewer comments 

and suggestions.

REFERENCES

Anderson, E. A., and N. H. Crawford, 1964: 

The synthesis of continuous snowmelt 

runoff hydrographs on a digital computer. 

Technical Report No. 36, Department of 

Civil Engineering, Stanford University, 

Stanford, California, 103 pp.

Anderson, E. A., 1968: Development and 

testing of snow pack energy balance 

equations. Water Resources Research, 4(1), 

19-37.

Anderson, E. A., 1973: National Weather 

Service river forecast system – snow 

accumulation and ablation model. NOAA 

Technical Memorandum NWS HYDRO-17, 

217 pp.

Burnash, R. J. C., R. L. Ferral, and R. A. 

McGuire, 1973: A generalized streamflow

simulation system – conceptual modeling 

for digital computers. U.S. Department of 

Commerce, National Weather Service and 

State of California, Department of Water 

Resources.

Cognitore, P., 2005: An investigation of 

Multi-sensor Precipitation Estimates (MPE) 

and operational use of MPE at the Middle 
22

Atlantic River Forecast Center (MARFC).

Eastern Region Technical Attachment, 

2005-03.

Linsley, R., M. Kohler, and J. Paulhus, 

1949: Runoff Relations. In: Applied 

Hydrology, Chapter 16, McGraw-Hill, New 

York, USA.

NWS, 2013: NWSRFS User’s Manual. 

[Available on-line from

http://www.nws.noaa.gov/oh/hrl/nwsrfs/user

s_manual/part2/_pdf/23apicont.pdf.]

Riverside Technology, inc. (RTi), 2009: 

Calibration for Portions of the Susquehanna 

River Basin. Task Completion Report Task 

8-0017.

Roe, J. M., C. Dietz, P. Restrepo, J. 

Halquist, R. Hartman, R. Horwood, B. 

Olsen, H. Opitz, R. Shedd, and E. Welles, 

2010: Introduction of NOAA’s Community 

Hydrologic Prediction System. Presented at 

the 90th American Meteorological Society 

(AMS) Annual Meeting, January 16-21, 

2010, Atlanta, GA, as part of the 26th 

Conference on Interactive Information and 

Processing Systems (IIPS) for Meteorology, 

Oceanography, and Hydrology.

Sittner, W. T., Schauss, C. E., and Monro, J. 

C., 1969: Continuous hydrograph synthesis 

with an API-type hydrologic model. Water 

Resources Research, 5(5), 1007-1022.

Smith, M. B., V. I. Koren, E. Welles, D. 

Wang, and Z. Zhang, 2000: Evaluation of 

the advantages of the continuous SAC-SMA 

model over an event API model. Presented 

at the 15th Conference on Hydrology, AMS, 

January 9-14, Long Beach, CA.

Susquehanna River Basin Commission 

(SRBC), 2012: Susquehanna River Basin 

Commission Information Sheet. [Available 

on-line from

http://www.srbc.net/pubinfo/factsheets.htm.]

Zhang, Y., Reed, S., and Kitzmiller, D.,

2011: Effects of retrospective gauge-based 

readjustment of multisensor precipitation 

estimates on hydrologic simulations.

Journal of Hydrometeorology, 12(3), 429-

443.