United States Department of Agriculture

Hydrologic Modeling for Flow-Ecology Science

in the Southeastern United States and Puerto Rico

Forest Service

Southern Research Station

e-General Technical Report SRS-246

January 2020
Peter V. Caldwell is a Research Hydrologist, U.S. Department of Agriculture Forest 

Service, Southern Research Station, Coweeta Hydrologic Laboratory, Otto, NC 

28763; Jonathan G. Kennen is the Ecological Water Science Lead, U.S. Department 

of the Interior, U.S. Geological Survey, National Water Census, Lawrenceville, 

NJ 08648; Ernie F. Hain is a PhD candidate, North Carolina State University, 

Department of Forestry and Environmental Resources, Center for Geospatial 

Analytics, Raleigh, NC 27695; Stacy A.C. Nelson is a Professor, North Carolina 

State University, Department of Forestry and Environmental Resources, Center 

for Geospatial Analytics, Raleigh, NC 27695; Ge Sun is a Research Hydrologist, 

U.S. Department of Agriculture Forest Service, Southern Research Station, Eastern 

Forest Environmental Threat Assessment Center, Research Triangle Park, NC 27709; 

and Steve G. McNulty is the Director, U.S. Department of Agriculture Southeast 

Climate Hub, Research Triangle Park, NC 27709.

Disclaimer: Any use of trade, firm, or product names is for descriptive purposes only 

and does not imply endorsement by the U.S. Government.

Photo Credit: Cover Photo: (Donald Lee Pardue, Flickr) The Haw River provides 

drinking water to nearly one million people living in and around the cities of 

Greensboro, Burlington, Chapel Hill, Cary, and Durham, and is home to a variety of 

fish and wildlife, including blue heron, bald eagle, beaver, deer, otter, largemouth 

and smallmouth bass, bowfin, crappie, carp, bluegill, and the endangered Cape Fear 

shiner and an assortment of rare freshwater mussel species. (https://www.

americanrivers.org/river/haw-river/)

January 2020

Southern Research Station

200 W.T. Weaver Blvd.

Asheville, NC 28804

www.srs.fs.usda.gov
Hydrologic Modeling 

for Flow-Ecology Science

in the Southeastern United States and Puerto Rico

Peter V. Caldwell, Jonathan G. Kennen, Ernie F. Hain, 

Stacy A.C. Nelson, Ge Sun, and Steve G. McNulty
ii

Abstract

An understanding of the applicability and utility of hydrologic models is 

critical to support the effective management of water resources throughout 

the Southeastern United States (SEUS) and Puerto Rico (PR). Hydrologic 

models have the capacity to provide an estimate of the quantity of available 

water at ungauged locations (i.e., areas of the country where a U.S. 

Geological Survey [USGS] continuous record gauge is not installed) and 

provide the baseline flow information necessary to develop the linkages 

between water availability and characteristics of streamflow that support 

ecological communities (i.e., support the development of flow-ecology 

response models). This report inventories and then directly examines and 

compares a subset of hydrologic models used to estimate streamflow at a 

number of gauged basins across the SEUS and PR. This effort was designed 

to evaluate, quantify, and compare the magnitude of error and to investigate 

the potential causes of error associated with predicted streamflows from 

seven hydrologic models of varying complexity and calibration strategy. 

This was accomplished by computing and then comparing classical 

hydrologic model fit statistics (e.g., mean bias, coefficient of determination 

[R2

], root mean squared error [RMSE], Nash-Sutcliffe Efficiency [NSE]) and 

understanding the bias in the prediction in these and a subset of ecologically 

relevant flow metrics (ERFMs). Additionally, streamflow predictions from 

a larger regional-scale hydrologic model were compared to those of several 

fine-scale hydrologic models under a range of hypothetical climate change 

scenarios to determine the range of predicted streamflow responses to 

fixed climate perturbations. A pilot study was conducted using predicted 

streamflow and boosted regression trees to develop a set of predictive 

flow-ecology response models to assess the potential change in fish species 

richness in the North Carolina Piedmont under several scenarios of water 

availability change. This report is intended to provide a general assessment 

of all the tools and techniques available to support hydrologic modeling 

for flow-ecology science in the SEUS and PR. It is our hope that the 

approach used herein to understand differences in streamflow predictions 

among a subset of hydrologic models that have been applied in the SEUS 

for developing flow-ecology response models will provide water resource 

managers and stakeholders with an informed pathway for developing the 

capacity to link streamflow and ecological response and an understanding of 

some of the limitations associated with these type of modeling efforts. 

Keywords: Ecological flows, fish species richness, flow alteration, flowecology models, hydrologic models, water supply, water withdrawals.
USDA Forest Service, Southern Research Station iii e-General Technical Report SRS-246

Table of Contents

CHAPTER 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Research Objectives..

Relationship to Other Studies . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Organization of this Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

CHAPTER 2 Inventory of Existing Hydrologic Models 

in the Southeastern United States and Puerto Rico..

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

CHAPTER 3 Evaluation and Comparison of Hydrologic 

Model Performance in Predicting Observed Streamflows 

in the Southeastern United States . . . . . . . . . . . . . . . . . . . . 10

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

CHAPTER 4 Feasibility of Combining Regionaland Local-Scale Models to Identify Unique Areas of 

Concern and Understand Fine-Scale Hydrologic 

Dynamics Under Climate Change . . . . . . . . . . . . . . . . . . . . . 38

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

CHAPTER 5 Using Regional-Scale Flow-Ecology 

Modeling to Identify Catchments Where Fish 

Assemblages are Most Vulnerable to Changes in 

Water Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

CHAPTER 6 Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

List of Acronyms and Abbreviations ..

USDA Forest Service, Southern Research Station 1 e-General Technical Report SRS-246

CHAPTER 1

Introduction

A

ssessing the impact of flow alteration on aquatic 

ecosystems has been identified as a critical area 

of research and a priority science need of the U.S. 

Department of the Interior Southeast Climate Adaptation 

Science Center (SCASC). Beginning in 2012, U.S. 

Department of Agriculture Forest Service scientists 

collaborated with scientists from the U.S. Geological 

Survey (USGS) and North Carolina State University in 

a SCASC-funded project to develop an inventory and 

evaluation of current efforts and knowledge gaps in 

hydrologic modeling for flow-ecology studies across the 

Southeastern United States (SEUS) and Puerto Rico (PR). 

To accomplish this goal, we synthesized and evaluated 

hydrologic modeling efforts in the SEUS (including 

all States of the Southeastern Association of Fish and 

Wildlife Agencies: Alabama, Arkansas, Florida, Georgia, 

Kentucky, Louisiana, Mississippi, Missouri, North 

Carolina, Oklahoma, South Carolina, Tennessee, Texas, 

Virginia, and West Virginia) and Puerto Rico. Because this 

modeling synthesis was performed comprehensively and 

using a consistent methodology, it will provide Landscape 

Conservation Cooperatives (LCCs) and resource managers 

with a useful database of who is doing what, where, how, 

and how well in terms of hydrologic modeling across the 

SEUS and PR. 

BACKGROUND

River flows are essential for sustaining the health of 

aquatic ecosystems and maintaining ecosystem services 

such as water supply for consumptive use. Human 

activities, including regulation by dams (Biemans 

and others 2011, Graf 1999, Poff and others 2007), 

withdrawals (Gerten and others 2008), interbasin 

transfers (Emanuel and others 2015, Jackson and others 

2001), and land cover change (Foley and others 2005) 

have significantly altered the magnitude and timing of 

river flows. The health and biotic condition of aquatic 

ecosystems have declined as a result (Carlisle and others 

2010, Dudgeon and others 2006, Poff and Zimmerman 

2010, USEPA 2011). In addition to anthropogenic 

hydrologic alterations, future changes in climate will 

likely further impact river flows (Georgakakos and 

others 2014). Assessing the effect of flow alteration 

on aquatic ecosystems has been identified as a critical 

area of research in the SEUS (Knight and others 2014, 

SALCC 2012, SARP 2004), nationally (Carlisle and others 

2010, Novak and others 2016), and abroad (Annear and 

others 2004; Arthington and others 2006, 2018; Poff and 

others 2010). The SEUS is recognized as one of the most 

ecologically rich areas in the world (Masters and others 

1998), making it imperative to assess ecological response 

to flow alteration. As a result, and considering recent 

droughts and interstate conflict over water availability 

issues, many States in the SEUS are investigating the 

implementation of regulatory controls on streamflow 

alteration in the interest of maintaining a balance between 

supporting healthy aquatic ecosystems and providing 

ample water supplies for human use (e.g., NCEFSAB 

2013). Reliable, multi-scale hydrologic and ecosystem 

modeling approaches are needed to accomplish this goal 

(Poff and others 2010). However, error (i.e., uncertainty 

or bias) in streamflow predictions by hydrologic models 

and predictions of ecological response to changes in flow 

regime with ecological models can be significant and may 

be compounding, thereby confounding the determination 

of environmental flow requirements and potentially 

exposing managers of water resources to litigation by the 

regulated community.

The Ecological Limits of Hydrologic Alteration (ELOHA) 

framework (Poff and others 2010) is often used as a basis 

for developing regional environmental flow requirements. 

One of the first steps emphasized in the ELOHA process is 

to develop a hydrologic foundation of simulated baseline 

and altered streamflow hydrographs. Hydrologic models 

are commonly used for this purpose because they have the 

ability to simulate streamflow at varying timescales under 

baseline conditions and an infinite number of scenarios 

of flow alteration. Many models have been employed in 

the SEUS to generate a hydrologic foundation on which 

to examine and test hypotheses of changes in water 

availability, land use, and climate. 

Major modeling efforts in the SEUS and PR include:

1. The South Atlantic LCC (SALLC) funded a project 

with The Nature Conservancy and Research Triangle 

Institute (RTI) to simulate daily flows in the SALCC 

at a very fine spatial resolution using a rainfall-runoff 

model (https://waterfall.rti.org/). 

2. The U.S. Geological Survey is developing multiresolution daily rainfall-runoff and statistical 

hydrologic models in the Apalachicola-ChattahoocheeFlint Basin for climate change and ecohydrological 

assessments as part of the Flint River Science Thrust 

Project, Southeast Regional Assessment Project (Dalton 
Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

2

and Jones 2010), and the National Water Census 

(https://www.usgs.gov/mission-areas/water-resources/

science/apalachicola-chattahoochee-flint-river-basinfocus-area-study?qt-science_center_objects=0#qtscience_center_objects) (Alley and others 2013). In 

addition, rainfall-runoff models are under development 

for Puerto Rico in support of the Caribbean LCC. 

3. The U.S. Environmental Protection Agency (USEPA) 

Global Change Research Program (http://www.

globalchange.gov/agency/environmental-protectionagency) has contracted with Tetra Tech and others to 

develop rainfall-runoff models to evaluate the potential 

effects of climate and land use change on the hydrology 

and water quality of 20 major U.S. drainage basins, 

including the Albemarle-Pamlico Basin, ApalachicolaChattahoochee-Flint (ACF) Basin, and a GeorgiaFlorida Coastal Plain basin (USEPA 2013).

4. The Georgia Water Resources Institute at Georgia 

Tech developed a decision-support system (http://

gwri.gatech.edu/research/GWRI/ACFS) including 

hydrologic models for the ACF Basin to evaluate water 

management alternatives under several climate change 

scenarios (Georgakakos and others 2010).

5. The U.S. Department of Agriculture Forest Service 

developed a national-scale water balance and flow 

routing model for the conterminous United States 

(https://forestthreats.org/research/tools/WaSSI) capable 

of predicting monthly streamflow as influenced by 

changes in climate, land use, and water withdrawals 

(Caldwell and others 2012; Sun and others 2008, 

2011b). The model is currently under development to 

predict monthly streamflow in Puerto Rico.

6. Many States have their own hydrologic modeling 

activities to develop water resource adaptation 

strategies for responding to environmental change by 

using various models (Davis 2011). For example, North 

Carolina is developing models for key basins to ensure 

sustainable water resources for the future (https://www.

ncwater.org/Data_and_Modeling/Tar/).

All of these approaches operate at different resolutions, 

extents, time periods, and model performance criteria, 

and take into account different aspects of environmental 

change (climate, land use, and water availability). A 

comprehensive synthesis of these efforts is needed so that 

managers may make informed decisions regarding model 

use and fully understand the uncertainty associated with 

their predictions of streamflow and ecological response. 

Regardless of the hydrologic model used, the model must 

be able to reasonably replicate streamflow observations 

at a monthly, daily, or sub-daily timescale to derive 

ecologically relevant flow metrics (ERFMs) representing 

the five primary components of the flow regime (i.e., 

magnitude, frequency, duration, timing, and rate of 

change) under historical and current conditions across 

points of interest for regulation. The model must also 

predict changes in these metrics as a result of changes in 

climate, land use, flow regulation, and water withdrawals. 

Due to the compounding nature of error in flow-ecology 

modeling, error in hydrologic model prediction of 

streamflow is often carried through the analysis and should 

be quantified and reported to avoid misinterpretation or 

misuse of modeling outcomes. 

Given the diversity of hydrologic models available for 

water supply and environmental flow studies, it would 

be useful for resource managers to have some sense 

of the relative error in streamflow predictions among 

commonly used hydrologic models in terms of classical 

fit statistics of streamflow observations (e.g., bias, NashSutcliffe Efficiency [NSE]) and ERFMs. Unfortunately, 

the determination as to which model is best suited for 

answering a particular ecological question or even whether 

one model is better suited for a specific region or for a 

specific portion of the flow regime (e.g., low flows) is 

hampered by a lack of comprehensive model comparison 

studies (Knight and others 2012). In addition, the type 

of hydrologic model, model inputs (e.g., climate, soils, 

land cover), and calibration strategy likely influence 

the capacity of a given hydrologic model to predict 

observed streamflow. 

Leveraging the benefits of utilizing large-scale models 

with smaller scale, high-resolution models concurrently, 

rather than using each approach in isolation, has the 

potential to allow more robust environmental change 

assessment studies that maintain a better balance between 

the availability of water to support aquatic assemblages 

while conserving water for long-term human needs across 

broad regions. For example, the Water Supply Stress Index 

(WaSSI) model (Caldwell and others 2012, Sun and others 

2011b) is a regional-scale monthly water balance and flow 

routing model that has been used to evaluate the effects 

of environmental change on water supply and river flows. 

The model is typically run uncalibrated using off-theshelf databases and thus could be used to assess broadscale environmental change and identify specific areas of 

concern (“hot spots”) where the combined effects of land 

cover change, climate change, and/or flow alteration may 

threaten water resources. Fine-scale, physically based 

models of higher temporal resolution (e.g., Hydrological 

Simulation Program-Fortran [HSPF], Precipitation-Runoff 

Modeling System [PRMS], Soil and Water Assessment 

Tool [SWAT], etc.) could then be applied to those areas of 

concern to provide higher resolution quantitative estimates 
USDA Forest Service, Southern Research Station e-General Technical Report SRS-246

CHAPTER 1

3

of changes in water supply and ERFMs using more sitespecific inputs. To apply such a multi-scale modeling 

approach, the variability of predicted streamflow response 

to different stressors across large- and fine-scale models 

must first be assessed. For example, for a given change 

in precipitation, do the models predict similar changes 

in streamflow?

RESEARCH OBJECTIVES

The objectives of this study were to (1) inventory existing 

hydrologic modeling efforts in the SEUS and PR, (2) 

evaluate and compare the performance of participating 

hydrologic models in predicting observed streamflows at 

multiple scales, (3) demonstrate the feasibility of using 

regional- and fine-scale models to identify unique areas of 

concern and understand fine-scale hydrologic dynamics for 

climate change assessment, respectively, and (4) conduct 

a pilot study at the regional scale using flow-ecology 

response modeling to assess the effect of changes in water 

availability in the North Carolina Piedmont based on a 

series of plausible change scenarios including climate, land 

use, and water withdrawals. 

For objective 1, modeling efforts in the region were 

inventoried by contacting Federal and State agencies, 

members of academia, and environmental consultants 

to create a database of models used in the SEUS and 

PR. Information collected included the organization 

performing the study, intended purpose, model framework, 

spatial extent, spatial and temporal resolution, time period 

simulated, model inputs, model outputs (e.g., flow, water 

quality, ecosystem response), elements of environmental 

change represented (e.g., climate change, land use change, 

withdrawals/flow regulation), and validation procedure, 

criteria, and results.

For objective 2, a subset of the model developers attended 

a workshop where attendees compared and contrasted the 

hydrologic output of both coarse-scale monthly models 

and fine-scale daily models, quantifying their ability to 

estimate observed flows over a common time period for 

selected basin(s) in the SEUS. We did not evaluate model 

applications in PR because no models had streamflow 

predictions at gauged locations at the time of this study.

For objective 3, streamflow predictions from WaSSI, a 

regional-scale model, were compared to those of several 

of the fine-scale models assessed in the model comparison 

workshop under a range of hypothetical climate change 

scenarios, to determine the range of predicted streamflow 

responses to fixed climate perturbations. Our working 

hypothesis was that similarly predicted streamflow 

changes under climate change scenarios among regionaland fine-scale models would provide evidence that 

these models could be used in combination to identify 

hot spots of concern and understand unique fine-scale 

hydrologic dynamics.

For objective 4, a set of predictive flow-ecology response 

models were developed that assess the potential change 

in fish species richness in the North Carolina Piedmont 

under several scenarios of water availability change 

including streamflow withdrawals, climate change, and 

land cover change.

RELATIONSHIP TO OTHER STUDIES

Through a partnership between the SALCC and the 

Southeast Aquatic Resources Partnership (SARP), an 

inventory of existing models in the SALCC region was 

completed (Davis 2011). The Appalachian LCC had 

a similar project with the New York Cooperative Fish 

and Wildlife Research Unit for the Appalachian LCC 

region (Fisher and others 2013). The USGS, as part of 

the WaterSMART Initiative National Water Census, led a 

multi-model comparison study in the Southeast which was 

designed to assess the relative uncertainty in hydrologic 

predictions among several daily and monthly time-step 

models (Farmer and others 2014). The project described 

in this report was intended to complement and more 

broadly encapsulate prior work in the SEUS, including 

the SALCC hydrologic model inventory, the Appalachian 

LCC hydrologic model inventory, and the National Water 

Census model comparison study, and provide additional 

insights through direct model comparisons, evaluation of 

model sensitivity, and the development of flow-ecology 

response models.

ORGANIZATION OF THIS REPORT

Chapters 2–5 of this report summarize the results 

of each of the research objectives described above. 

The chapters were written to be “self-contained” and 

can be read without referring to other chapters. This 

format was selected to make reading of the report 

easier. Detailed results, databases, and links to reports 

may be found online at https://cascprojects.org/#/

project/4f8c6557e4b0546c0c397b4c/

5016cacde4b06fb5ce8b7371. 
4

CHAPTER 2

Inventory of Existing Hydrologic Models in 

the Southeastern United States and Puerto Rico

INTRODUCTION

There are many researchers, water resource professionals, 

and environmental flow practitioners developing and 

applying hydrologic models in the Southeastern United 

States (SEUS) and Puerto Rico (PR). However, there are 

few databases that describe the various modeling efforts 

underway across the region so that resource managers may 

have access to model outputs to aid in addressing water 

resource problems. To meet this need, an inventory of 

current efforts and knowledge gaps in hydrologic modeling 

for flow-ecology science and environmental change 

assessment studies across the region was created. This 

modeling inventory was performed as comprehensively 

as possible and used a consistent methodology, and thus 

it will provide Landscape Conservation Cooperatives 

(LCCs) and resource managers with a useful database of 

organizations developing hydrologic models for assessing 

environmental change effects across the SEUS and PR. 

METHODS

An inventory of existing hydrologic models in the SEUS 

and PR was compiled by distributing a brief questionnaire 

(table 2.1) to contacts at State and Federal agencies, 

academic institutions, and environmental consultants 

throughout the region (table 2.2). An initial list of contacts 

was developed via the professional knowledge of project 

participants and by using the SEUS Hydrologic Model 

Assessment compiled by Southeast Aquatic Resources 

Partnership (SARP) (https://southeastaquatics.net/

resources/pdfs/Hydrologic%20Model%20Assessment%20

041411.pdf/). The information requested from these 

contacts included model developer, intended purpose, 

model framework, spatial extent, spatial and temporal 

resolution, time period simulated, model inputs, model 

outputs (e.g., flow, water quality, ecosystem response), 

elements of environmental change represented (e.g., 

climate change, land use change, withdrawals/flow 

regulation change), and validation procedure, criteria, and 

results. Questionnaire interviews were primarily conducted 

by phone, with the exception of a few individuals who 

responded to the questionnaire by email. Questionnaire 

results were stored in a Microsoft® Excel spreadsheet.

RESULTS

The results of the modeling inventory are available 

at the project web page at https://cascprojects.org/#/

project/4f8c6557e4b0546c0c397b4c/5016

cacde4b06fb5ce8b7371. The model inventory contact 

list included 95 individuals applying hydrologic 

models throughout the SEUS and PR. Of the 64 

unique organizations solicited, 19 represented Federal 

agencies, 11 represented State agencies, 32 represented 

universities, and 2 represented private sector organizations 

(environmental consultants). Twenty of the individuals 

agreed to be interviewed or were participants in the 

model comparison workshop, with two of the individuals 

using two different models. Respondents used multiple 

hydrologic models and were working throughout the 

SEUS and PR (table 2.3). Of these, 16 used rainfall-runoff 

models (i.e., models that predict streamflow using climate 

inputs including rainfall), while four of the respondents 

used statistical models (i.e., estimates of streamflow 

derived empirically using streamflow records from 

gauged basins). Thirteen of the respondents represented 

government agencies, while five and two represented 

universities and private sector organizations, respectively. 

Model uses ranged from estimating streamflow under 

scenarios of changing climate and water use to predicting 

effects on water quality, aquatic ecosystems, and water 

availability for humans. Basins in Alabama, Georgia, and 

North Carolina were the best represented in the survey 

results (7 or more of the 20 respondents); however, a 

large proportion of the basins simulated in these States 

were restricted to the Apalachicola-Chattahoochee-Flint 

(ACF) Basin. Basins in Arkansas, Kentucky, Louisiana, 

Mississippi, Missouri, Oklahoma, Texas, West Virginia, 

and Puerto Rico were least represented (three or fewer 

respondents). While these results are based on a subsample 

of modeling efforts in the region, they may suggest areas 

that are less studied, and thus potential environmental 

change effects on streamflow may be less understood.
USDA Forest Service, Southern Research Station 5 e-General Technical Report SRS-246

CHAPTER 2

Table 2.1—Questions from hydrologic model inventory questionnaire

1. What is the name of your model (e.g., WaSSI, PRMS)?

2. Who developed the model (e.g., individual researcher, agency initiative)?

3. Who applied the model?

4. What is the intended purpose of your model (e.g., to simulate daily flows at a very fine spatial 

resolution, water supply accounting, ecological flow assessment, water quality)?

5. Please describe the model framework.

6. What is the spatial extent of your model (e.g., Hydrologic Unit Code [HUC], river basin, etc.)? Do you 

have an attributed shapefile available that you could share with us?

7. What is the spatial resolution of your model (e.g., grid cell, reach, basin, ecoregion) and approximate 

size of model units (e.g., ha, km2

)?

8. What is the temporal resolution of your model (e.g., hourly, daily, monthly)?

9. What is the time period represented by your model (e.g., start year and end year simulated)?

10. What inputs does your model require (e.g., weather parameters, soils, land cover, water withdrawals, 

etc.), and how are they derived (calibrated, remote sensing, etc.)?

11. What are the model outputs (e.g., flow, water quality, ecosystem response)?

12. What elements of environmental change are represented by your model, if any (e.g., climate, land 

use change, withdrawals/flow regulation)?

13. Please explain the procedure, criteria, and results of your model validation (e.g., specific U.S. 

Geological Survey gauges, other flow data)?

14. What time scale was validated (e.g., mean annual, annual, seasonal, monthly, daily, hourly)?

15. What fit statistics were used (e.g., bias, R2

, Nash-Sutcliffe Efficiency, etc.), and at what level was the 

model performance considered satisfactory?

16. How can we use your model (i.e., Is it downloadable, proprietary, publicly available?)?

17. May we contact you again with further questions/technical assistance if necessary?

18. Can you recommend other professionals we should contact regarding hydrologic modeling, 

especially flow ecology/ecohydrology in the Southeast/Puerto Rico? 

19. Has your model ever been used in the Apalachicola-Chattahoochee-Flint (ACF) Basin? Is there any 

reason why your model would not be an appropriate choice to use in the ACF Basin, or, alternatively, 

is there any reason why your model would be an ideal choice to use in the ACF Basin?
6

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Table 2.2—List of Federal and State agencies, academic institutions, and environmental consultants that were 

contacted for an inventory of existing hydrologic models 

Organizationa Location Type

Total 

solicitedb

Total 

responsesb

Alabama Department of Economic and Community Affairs, Office of 

 Water Resources

Alabama State 1 1

The University of Alabama in Huntsville, Department of Civil and 

 Environmental Engineering

Alabama University 1 0

Auburn University, Alabama Water Resources Research Institute Alabama University 1 0

Auburn University, Department of Biosystems Engineering Alabama University 1 0

Auburn University, School of Forestry and Wildlife Sciences Alabama University 1 0

The University of Alabama in Huntsville, Department of Atmospheric Science Alabama University 2 0

Arkansas Department of Environmental Quality, Water Division Arkansas State 2 0

U.S. Geological Survey, Arkansas Water Science Center Arkansas Federal 1 0

Purdue University, Department of Agricultural & Biological Engineering Arkansas University 1 1

Arkansas Cooperative Fish and Wildlife Research Unit Arkansas State 1 1

University of Arkansas, Division of Agriculture, Arkansas Water Resources Center Arkansas University 2 0

University of Arkansas, Department of Biological and Agricultural Engineering Arkansas University 2 0

University of Arkansas, Department of Crop, Soil, and Environmental Science Arkansas University 1 0

U.S. Geological Survey, Global Change Research Program Colorado Federal 1 2

University of Florida, Water Resources Research Center Florida University 2 0

U.S. Geological Survey, Caribbean-Florida Water Science Center Florida Federal 2 0

Georgia Tech, Georgia Water Resources Institute Georgia University 1 0

U.S. Geological Survey, Global Change Research Program Georgia Federal 1 1

U.S. Department of Agriculture Agricultural Research Service, Southeast 

 Watershed Research

Georgia Federal 1 0

U.S. Geological Survey, South Atlantic Water Science Center Georgia Federal 1 0

University of Notre Dame, Department of Biological Sciences Indiana University 1 1

U.S. Geological Survey, Ohio-Kentucky-Indiana Water Science Center Kentucky Federal 2 2

Kentucky Energy and Environment Cabinet, Department for Environmental 

 Protection, Division of Water 

Kentucky State 3 0

University of Kentucky, Kentucky Water Resources Research Institute Kentucky University 2 0

U.S. Geological Survey, Lower Mississippi-Gulf Water Science Center Louisiana Federal 2 0

Louisiana State University, Louisiana Water Resources Research Institute Louisiana University 3 0

Mississippi State University, Mississippi Water Resources Research Institute Mississippi University 2 0

Mississippi State University, Department of Civil and Environmental Engineering Mississippi University 1 1

U.S. Geological Survey, Lower Mississippi-Gulf Water Science Center Mississippi Federal 1 0

Missouri Department of Natural Resources, Water Resources Center Missouri State 1 0

U.S. Geological Survey, Central Midwest Water Science Center Missouri Federal 3 0

University of Missouri, Missouri Water Resources Research Center Missouri University 1 0

U.S. Geological Survey, South Atlantic Water Science Center North Carolina Federal 1 0

North Carolina Department of Environmental Quality, Division of Water Resources North Carolina State 1 0

North Carolina State University, Department of Civil, Construction, and 

 Environmental Engineering

North Carolina University 2 0

Water Resources Research Institute of the University of North Carolina System North Carolina University 3 0

(continued on next page)
USDA Forest Service, Southern Research Station 7 e-General Technical Report SRS-246

CHAPTER 2

Organizationa Location Type

Total 

solicitedb

Total 

responsesb

Duke University, Nicholas School of the Environment North Carolina University 1 1

U.S. Department of Agriculture Forest Service, Eastern Forest Environmental Threat 

 Assessment Center

North Carolina Federal 1 1

Tetra Tech North Carolina Consultant 1 2

Research Triangle Institute North Carolina Consultant 1 1

North Carolina State University, Department of Biological Sciences North Carolina University 1 0

Oklahoma Water Resources Board  Oklahoma State 2 1

Oklahoma State University, Oklahoma Water Resources Center Oklahoma University 2 0

Oklahoma State University, Department of Natural Resource Ecology and 

 Management 

Oklahoma University 1 0

U.S. Geological Survey, Oklahoma Water Science Center Oklahoma Federal 1 0

South Carolina Department of Natural Resources, Hydrology Section South Carolina State 1 0

U.S. Geological Survey, South Atlantic Water Science Center South Carolina Federal 1 1

Clemson University, South Carolina Water Resources Center South Carolina University 3 0

University of South Carolina, School of the Earth, Ocean and Environment South Carolina University 1 1

Tennessee Department of Environment & Conservation, Division of 

 Water Resources

Tennessee State 1 0

U.S. Geological Survey, Lower Mississippi-Gulf Water Science Center Tennessee Federal 1 1

The University of Tennessee, Knoxville, Tennessee Water Resources 

 Research Center

Tennessee University 1 0

Tennessee Technical University, Center for the Management, Utilization and 

 Protection of Water Resources

Tennessee University 1 0

Tennessee Technical University, Department of Civil & Environmental Engineering Tennessee University 1 0

Texas Commission on Environmental Quality, Office of Water, Water 

 Availability Division

Texas State 1 0

U.S. Geological Survey, Texas Water Science Center Texas Federal 5 1

Texas Water Resources Institute Texas University 2 0

U.S. Army Corps of Engineers, System-Wide Water Resources Program Texas Federal 1 0

Texas A&M University, Spatial Sciences Laboratory Texas University 1 0

Texas A&M University, Department of Civil Engineering, Environmental and Water 

 Resources Engineering Division

Texas University 1 0

U.S. Geological Survey, Virginia and West Virginia Water Science Center Virginia Federal 2 1

Virginia Tech, Virginia Water Resources Research Center Virginia University 2 0

West Virginia Department of Environmental Protection, Division of Water and 

 Waste Management

West Virginia State 1 0

U.S. Geological Survey, Virginia and West Virginia Water Science Center West Virginia Federal 2 1

aOrganizations that were successfully contacted were asked to provide answers to a simple questionnaire (see table 2.1) regarding their hydrologic 

modeling efforts. 

b Total solicited = the number of individuals solicited within each organization; Total responses = the number of modeling efforts described by those 

individuals who were successfully contacted within each organization. Where total responses are greater than the total solicited, there was more than one 

modeling effort described by one or more of the individuals contacted in that organization.

Table 2.2 (continued)—List of Federal and State agencies, academic institutions, and environmental consultants 

that were contacted for an inventory of existing hydrologic models 
8

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Table 2.3—Summary of survey responses and a description of the general purpose of flow models used in 

Southeastern United States and Puerto Rico modeling studies 

Location of basins simulated

Organization Model type

ALARFLGAKYLAMSMONCOKSCTNTXVAWVPRUse

USGS Ohio-Kentucky- 

 Indiana Water 

 Science Center

Rainfall-runoff 

(TOPMODEL) X

Assess climate change effects 

on streamflow.

USGS Ohio-Kentucky-

 Indiana Water 

 Science Center

Rainfall-runoff 

(TOPMODEL)

X

Provide historic and baseline 

streamflow hydrographs for 

several USGS gauge locations in 

the Apalachicola-ChattahoocheeFlint (ACF) Basin. 

Mississippi 

 State University, 

 Department of Civil 

 and Environmental 

 Engineering

Rainfall-runoff 

(HSPF)

X X X

Hydrological assessment of 

listed watersheds

USGS Lower 

 Mississippi-Gulf 

 Water Science Center

Rainfall-runoff 

(PRMS) X X X

Estimate flow at ungauged 

tributaries.

USGS Lower 

 Mississippi-Gulf 

 Water Science Center

Statistical 

(regional linear 

multivariate 

regression)

X X X X

Predict ungauged stream 

characteristics; employ 

predictions to determine their 

effect on documented fish 

communities. 

Arkansas Cooperative 

 Fish & Wildlife 

 Research Unit

Statistical 

(decision trees 

and random 

forest models)

X X X

Impacts on fish and invertebrates 

USGS Virginia and 

 West Virginia Water 

 Science Center

Rainfall-runoff 

(HSPF) X

Determine flow statistics under 

climate scenarios. Identify areas 

where mussel thresholds will be 

crossed under climate change.

USGS Virginia and 

 West Virginia Water 

 Science Center

Rainfall-runoff 

(HSPF) X

Cumulative Hydrologic Impact 

Assessment for coal mining.

Duke University, 

 Nicholas School of 

 the Environment

Rainfall-runoff 

(GR4J) X

Effect of doubling CO2

 on 

streamflow

USGS South Atlantic 

 Water Science Center

Statistical 

(Calibrated 

Artificial Neural 

Network)

X X

Simulate salinity levels for water 

utilities’ relicensing purposes.

USGS Texas Water 

 Science Centera

Rainfall-runoff

(RRAWFLOW)

Represent changes in response 

to climate but could also be set 

up to simulate the effects of 

groundwater withdrawals.

University of South 

 Carolina, School of 

 the Earth, Ocean and 

 Environment

Rainfall-runoff 

(HSPF) X X

Climate, land use, and water 

management effects on future 

streamflow and water quality

(continued on next page)
USDA Forest Service, Southern Research Station 9 e-General Technical Report SRS-246

CHAPTER 2

Location of basins simulated

Organization Model type

ALARFLGAKYLAMSMONCOKSCTNTXVAWVPRUse

Oklahoma Water 

 Resources Board

Statistical 

(CRAM) X

Administration of water rights; 

diversion of surface water and 

estimates if demands are met

University of Notre 

 Dame, Department of 

 Biological Sciencesa

Rainfall-runoff 

(SWAT)

Both climate and land use change 

scenarios can be represented.

Purdue University, 

 Department of 

 Agricultural & 

 Biological Engineering

Rainfall-runoff 

(SWAT) X

Quantify current and future 

ecosystem services and examine 

changes as a result of growing 

demand for biofuels.

USDA Forest Service, 

 Eastern Forest 

 Environmental Threat 

 Assessment Center

Rainfall-runoff 

(WaSSI) X X X X X X X X X X X X X X X X

Climate, land use, population 

growth

Tetra Tech Rainfall-runoff 

(HSPF & 

SWAT) X X X X X X

Assess the sensitivity of 

streamflow, nutrient, and 

sediment loads to climate change 

and urban development.

Research Triangle 

 Institute

Rainfall-runoff 

(GWLF)

X X X X X X

Hydrologic foundation 

development, resiliency, 

ecological flow development, 

and climate and land use change 

analyses

USGS Global Change 

 Research Program 

 (Colorado)

Rainfall-runoff 

(PRMS & 

MWBM)

X X X X X X X X Provide daily or monthly time 

series of streamflows at several 

sites across the Southeast as part 

of a USGS National Water Census 

study.

USGS Global Change 

 Research Program 

 (Georgia)

Rainfall-runoff 

(PRMS)

X X X Understand the effect of climate 

change on ecosystem responses 

as part of the USGS Southeast 

Regional Assessment Project.

a

 Respondent was model developer; their response did not describe application of the model.

CRAM = a proprietary network flow model used to simulate water resources systems; GR4J = a lumped bucket-type daily rainfall-runoff model; 

GWLF = Generalized Watershed Loading Function; HSPF = Hydrological Simulation Program-Fortran; MWBM = Monthly Water Balance Model; PRMS = 

Precipitation-Runoff Modeling System; RRAWFLOW = Rainfall-Response Aquifer and Watershed Flow Model; SWAT = Soil and Water Assessment Tool; 

TOPMODEL = physically based, semi-distributed topographical watershed model; USDA = U.S. Department of Agriculture; USGS = U.S. Geological 

Survey; WaSSI = Water Supply Stress Index model.

Table 2.3 (continued)—Summary of survey responses and a description of the general purpose of flow models 

used in Southeastern United States and Puerto Rico modeling studies 

CONCLUSIONS

Here we created an inventory of existing hydrologic 

modeling efforts in the SEUS and PR. The 22 individuals 

interviewed developed and used hydrologic models 

to answer broad questions regarding the impacts of 

environmental change on water resources. With the rapid 

pace of computing technology and growth of modeling 

approaches, as well as changing threats to watersheds 

across the landscape, it is expected that this model 

inventory will continue to evolve and thus represents a 

snapshot of approaches and knowledge gaps in hydrologic 

modeling for flow-ecology science and environmental 

change assessment studies across the SEUS and PR. 
10

CHAPTER 3

Evaluation and Comparison of Hydrologic Model 

Performance in Predicting Observed Streamflows in 

the Southeastern United States

INTRODUCTION

Hydrologic models are commonly used to develop a 

hydrologic foundation (Poff and others 2010) because 

they have the ability to simulate monthly, daily, or 

sub-daily streamflow under baseline conditions and an 

infinite number of scenarios of flow alteration. Available 

hydrologic models vary in their levels of complexity, 

temporal and spatial resolution, and required level of 

calibration. Detailed and highly parameterized fineresolution models (e.g., distributed physically based 

watershed and rainfall-runoff models) are well suited for 

smaller domains but can be computationally expensive 

and difficult to parameterize at a large scale. Easily 

parameterized models (e.g., lumped regional models) are 

useful for assessing broad implications of streamflow 

alteration at a large scale and identifying potential waterlimited areas (or “hot spots”) but may have difficulty 

resolving unique sub-watershed physical processes and 

anthropogenic effects. Regardless of the hydrologic model 

used, the model must be able to reasonably replicate 

streamflow observations and ecologically relevant flow 

metrics (ERFMs) describing the five primary components 

of the flow regime (i.e., magnitude, frequency, duration, 

timing, and rate of change) under historical and current 

conditions across points of interest for regulation. The 

model must also predict changes in these metrics as a 

result of changes in climate, land use, flow regulation, 

and water withdrawals. Error (i.e., uncertainty or bias) in 

hydrologic model prediction of streamflow and ecological 

model prediction of ecosystem response to changes in 

flow (e.g., development of flow-ecology response models) 

is compounding; therefore, error in hydrologic model 

prediction of streamflow is often carried through the 

analysis and should be quantified and reported to reduce 

the probability of spurious relations and misinterpretation 

of modeling outcomes. 

Given the number of choices of hydrologic models for 

water supply and environmental flow studies (see chap. 2), 

it would be useful for resource managers to have some 

sense of the relative error in streamflow predictions 

among commonly used hydrologic models in terms of 

classical fit statistics of streamflow observations (e.g., 

bias, Nash- Sutcliffe Efficiency [NSE]) and ERFMs. 

Unfortunately, the determination as to which model is 

best suited for answering a particular ecological question 

or even whether one model is better suited for a specific 

region or for a specific portion of the flow regime (e.g., 

low flows) is hampered by a lack of comprehensive model 

comparison studies (Knight and others 2012). In addition 

to the type of hydrologic model, model inputs (e.g., 

climate, soils, land cover, etc.), calibration strategy, and the 

experience of the modeler will influence the capacity of a 

given hydrologic model to predict observed streamflow. 

The aim of this study was to address gaps in our 

understanding of differences among hydrologic models 

when applied to 195 U.S. Geological Survey (USGS) 

continuous record gauging stations in the Southeastern 

United States (SEUS). We did not evaluate model 

applications in Puerto Rico (PR) because, at the time of 

this study, none of the models were sufficiently developed 

to provide streamflow predictions at gauged locations 

in PR. Since the completion of this work, the WaSSI 

model evaluated here was parameterized to predict 

monthly streamflow in PR (Cohen and others 2017, 

Zhang and others 2018). It was not the intent of this 

study to identify any specific model that is better suited 

for streamflow prediction over another because such 

differences are as likely to be related to model calibration 

strategy, experience, and personal preference as they are 

to differences in model structure. Rather, the overarching 

goal of this investigation was to quantify and compare 

the magnitude and investigate the potential causes of 

error associated with predicted streamflows from seven 

hydrologic models of varying complexity and calibration 

strategy by computing classical hydrologic model fit 

statistics (e.g., mean bias, coefficient of determination 

[R2

], root mean squared error [RMSE], and NSE) and 

bias in the prediction of ERFMs. In addition, we tested 

the hypotheses that (1) regional-scale hydrologic models 

would, in general, have poorer predictive capacity and 

higher levels of uncertainty than fine-scale models; and 

(2) models with higher levels of calibration would perform 

better than those with less calibration.
USDA Forest Service, Southern Research Station 11 e-General Technical Report SRS-246

CHAPTER 3

METHODS

Model Comparison Workshop

A model comparison workshop was held on February 12, 

2013 in the Department of Forestry and Environmental 

Resources Jordan Hall at North Carolina State University. 

Workshop attendees (table 3.1; fig. 3.1) included a subset 

of the developers presented in the model inventory 

(chap. 2). The workshop included an overview of the 

project, a statement from Adam Terando (U.S. Department 

of the Interior Southeast Climate Adaptation Science 

Center [SCASC]) providing an overview of the 

ecohydrology research priority and relevance to the 

SCASC mission, an overview of the National Water 

Census model comparison study by Julie Kiang (USGS 

Office of Surface Water), descriptions of modeling work 

by all modelers, presentation of statistical comparisons 

of predicted streamflow to observations at USGS gauges, 

and a discussion comparing and contrasting modeling 

approaches and how they relate to the ability to predict 

observed streamflow. 

Model Descriptions

Specific models discussed and compared in the 

workshop were the Hydrological Simulation ProgramFortran (HSPF); the USGS Monthly Water Balance 

Model (MWBM); two parameterizations of the USGS 

Precipitation-Runoff Modeling System (PRMS) model; 

the Soil and Water Assessment Tool (SWAT); three 

parameterizations of the Water Availability Tool for 

Environmental Resources (WATER), based on the USGS 

TOPMODEL (a physically based, semi-distributed 

topographical watershed model); the Generalized 

Table 3.1—Model comparison workshop attendees, the organizations they represent, and the hydrologic 

models used 

Attendee Organization Hydrologic model(s)

Jon Butcher Tetra Tech, Research Triangle Park, NC SWAT, HSPF

Peter Caldwella U.S. Department of Agriculture Forest Service, Eastern Forest 

Environmental Threat Assessment Center, Raleigh, NC

WaSSI

Michele Eddy Research Triangle Institute, Research Triangle Park, NC WaterFALL® (employs 

updated version of GWLF)

Lauren Hay U.S. Geological Survey, Global Change Research Program, Lakewood, CO PRMS, MWBM

Ernie Haina, b North Carolina State University, Department of Forestry and 

Environmental Resources, Raleigh, NC —

Jonathan Kennena, b U.S. Geological Survey, National Water Census, Lawrenceville, NJ —

Julie Kiang U.S. Geological Survey, Water Mission Area, Analysis and Prediction 

Branch, Reston, VA —

Jacob LaFontaine U.S. Geological Survey, Global Change Research Program, Atlanta, GA PRMS

Steven McNultya, b U.S. Department of Agriculture Forest Service, Eastern Forest 

Environmental Threat Assessment Center, Raleigh, NC —

Stacy Nelsona, b North Carolina State University, Department of Forestry and 

Environmental Resources, Raleigh, NC —

Catalina Segurab North Carolina State University, Department of Marine, Earth, and 

Atmospheric Sciences, Raleigh, NC —

Timothy Shortleya, b North Carolina State University, Department of Forestry and 

Environmental Resources, Raleigh, NC —

Ge Suna U.S. Department of Agriculture Forest Service, Eastern Forest 

Environmental Threat Assessment Center, Raleigh, NC

WaSSI

Tanja Williamson U.S. Geological Survey, Ohio-Kentucky-Indiana Water Science Center, 

currently in Lawrenceville, NJ

WATER (based on 

TOPMODEL)

a

 Project team member.

b

 Workshop attendees shown without an associated hydrologic model were project team members and/or contributors who did not provide 

model outputs for the comparison study.

GWLF = Generalized Watershed Loading Function; HSPF = Hydrological Simulation Program-Fortran; MWBM = Monthly Water Balance Model; 

PRMS = Precipitation-Runoff Modeling System; SWAT = Soil and Water Assessment Tool; TOPMODEL = physically based, semi-distributed 

topographical watershed model; WaSSI = Water Supply Stress Index model; WATER = Water Availability Tool for Environmental Resources; 

WaterFALL® = Watershed Flow and Allocation modeling system using NHDPlus.
12

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Figure 3.1—The model comparison workshop was a daylong event hosted by North Carolina State 

University. This workshop allowed hydrologic modelers from multiple organizations across the SEUS 

to meet in a single location and discuss similarities and differences in modeling approaches, input 

data sources, calibration strategies, and model performance. (Photo by Ge Sun, USDA Forest Service)

Watershed Loading Function (GWLF)-based WaterFALL®

model developed by Research Triangle Institute (RTI); 

and the U.S. Department of Agriculture (USDA) Forest 

Service Water Supply Stress Index (WaSSI) model 

(table 3.2). The MWBM and WaSSI models are regional, 

large-scale monthly streamflow models, while HSPF, 

PRMS, SWAT, WATER, and WaterFALL® are more 

complex highly parameterized, fine-scale daily or hourly 

streamflow models. All of the models evaluated in this 

study were developed by different organizations, for 

different purposes, and calibrated to different degrees 

using different objective functions as described below. 

HSPF and SWAT

Both the HSPF and SWAT models used in this study 

were implemented by Tetra Tech as part of a larger study 

to characterize the sensitivity of streamflow, nutrient 

loading, and sediment loading to a range of potential 

mid-21st century climate futures in 20 large U.S. drainage 

basins (Johnson and others 2012, USEPA 2013). Model 

descriptions and information pertinent to this application 

are detailed below.

HSPF—The HSPF (Bicknell and others 2001, 2005) is 

a hydrology and water quality model commonly used for 

determination of Total Maximum Daily Loads to receiving 

waters in response to the Clean Water Act. HSPF is a 

well-documented watershed model that computes the 

water balance based on the Stanford Watershed Model 

(Crawford and Linsley 1966) in multiple surface and 

subsurface layers at an hourly time step. The water balance 

is simulated based on Philip’s infiltration (Bicknell and 

others 2001, 2005) coupled with multiple surface and 

subsurface stores (interception storage, surface storage, 

upper zone soil storage, lower zone soil storage, active 

groundwater, and inactive [deep] groundwater). Individual 

land units within a sub-basin are represented using a 

hydrologic response unit (HRU) approach that combines 

an overlay of land cover, soil, and slope characteristics. 

The stream network links the surface runoff and 

groundwater flow contributions from each of the HRUs 

and routes them through water bodies. The stream model 

includes precipitation (PPT) and evaporation from the 

water surfaces as well as streamflow contributions from 

the watershed, tributaries, and upstream stream reaches. 

SWAT—SWAT was developed to simulate the effects 

of land management practices on water, sediment, 

and agricultural chemical yields in large, complex 

watersheds with varying soils, land use, and management 

conditions over long periods of time (Neitsch and others 

2005). SWAT requires data inputs for weather, soils, 
USDA Forest Service, Southern Research Station 13 e-General Technical Report SRS-246

CHAPTER 3

Table 3.2—General summary of model attributes 

Model

Time 

step Spatial resolution

Withdrawals, 

flow regulation 

simulated Land cover input

Potential 

evapotranspiration 

method Climate input

HSPF hourly HUC-10

(~410 km2

)

yes 2001 NLCD

(Homer and others 

2007)

Penman-Monteith 

(Monteith 1965, 

Jensen and others 

1990)

Station 

observations 

(USEPA 2008)

MWBM monthly Aggregated 

NHDPlus 

catchments

(~78 km2

)

no N/A Hamon

(Hamon 1963)

PRISM

(PRISM Climate 

Group 2013)

PRMS-SERAP daily HRU

(~200 km2

)

no 2001 NLCD

(Homer and others 

2007)

Jensen-Haise 

(Jensen and Haise 

1963)

Maurer

(Maurer and 

others 2002)

PRMS-DAYMET daily Aggregated 

NHDPlus 

catchments

(~78 km2

)

no 2001 NLCD

(Homer and others 

2007)

Jensen-Haise 

(Jensen and Haise 

1963)

DAYMET 

(Thornton and 

others 2013)

SWAT daily HUC-10

(~410 km2

)

yes 2001 NLCD

(Homer and others 

2007)

Penman-Monteith 

(Monteith 1965, 

Jensen and others 

1990)

Station 

observations 

(USEPA 2008)

WATER IDW 1992 daily 10 m x 10 m no 1992 NLCD

(Volgelmann and 

others 2001)

Hamon

(Hamon 1963)

IDW

(Hay and others 

2002)

WATER IDW 2006 daily 10 m x 10 m no 2006 NLCD

(Fry and others 2011)

Hamon

(Hamon 1963)

IDW

(Hay and others 

2002)

WATER NR 2006 daily 10 m x 10 m no 2006 NLCD

(Fry and others 2011)

Hamon

(Hamon 1963)

NEXRAD

WaterFALL® daily NHDPlus 

catchment

(~1.0 km2

)

no ca. 1970s USGS 

GIRAS (Price and 

others 2006)

Hamon

(Hamon 1963)

USDA

(Di Luzio and 

others 2008)

WaSSI monthly HUC-12

(~80 km2

)

no 2006 NLCD

(Fry and others 2011)

Function of leaf 

area, PPT, Hamon 

(Sun and others 

2011b)

PRISM

(PRISM Climate 

Group 2013)

DAYMET = Daily Meteorological Data; GIRAS = Geographic Information Retrieval and Analysis System; HRU = Hydrologic Response Unit; HSPF = 

Hydrological Simulation Program-Fortran; HUC = Hydrologic Unit Code; IDW = Inverse Distance Weighted; MWBM = Monthly Water Balance Model; 

NEXRAD = Next-Generation Radar; NHD = National Hydrography Dataset; NLCD = National Land Cover Database; NR = Next-Generation Radar; PPT = 

precipitation; PRISM = Parameter-elevation Relationships on Independent Slopes Model; PRMS = Precipitation-Runoff Modeling System; SERAP = 

Southeast Regional Assessment Project; SWAT = Soil and Water Assessment Tool; USDA = U.S. Department of Agriculture; WaSSI = Water Supply 

Stress Index model; WATER = Water Availability Tool for Environmental Resources; WaterFALL® = Watershed Flow and Allocation modeling system 

using NHDPlus.

topography, vegetation, and land use to model water and 

sediment movement, nutrient cycling, and numerous 

other watershed processes. SWAT (as implemented 

here) uses the curve number approach (USDA Soil 

Conservation Service 1972) to estimate surface runoff 

and then completes the water balance through simulation 

of subsurface flows, evapotranspiration (ET), soil 

storages, and deep seepage losses at the daily time step. 

The curve number is estimated as a function of land use, 

cover, condition, hydrologic soil group, and antecedent 

soil moisture. 

HSPF and SWAT for this application—For both models, 

the 20 larger watersheds were divided into a series of 

sub-basins at approximately the Hydrologic Unit Code 

(HUC) 10-digit scale, representing the drainage areas that 

contribute to each of the stream reaches. Both the HSPF 

and SWAT models used the 2001 National Land Cover 

Database (NLCD) (Homer and others 2007) to characterize 

the land surface. For HSPF, soils are distinguished on 

the basis of hydrologic soil group (HSG) as defined in 

the State Soil Geographic (STATSGO) database (USDA 

NRCS 2012) soil coverages. The HRU definitions for 

SWAT in this application use the same land cover as HSPF 

but distinguish soils based on STATSGO’s dominant soil 
14

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

classification, not just HSG. Withdrawals were included 

if they resulted in a modification of flow at downstream 

gauges on the order of 10 percent or more. Time series 

of observed PPT and air temperature (TEMP; hourly 

for HSPF, daily for SWAT) from 37 weather stations 

in the Apalachicola-Chattahoochee-Flint (ACF) Basin 

were obtained from the 2006 BASINS 4 Meteorological 

Database (USEPA 2008). Potential evapotranspiration 

(PET) for both HSPF and SWAT was computed using the 

Penman-Monteith energy balance method (Jensen and 

others 1990, Monteith 1965) using solar radiation, wind 

movement, cloud cover, and relative humidity estimated 

using the SWAT weather generator. A full energy balance 

approach was used because the focus of the study is 

on evaluating potential response to future climates, in 

which the relations between different energy inputs may 

change, even though a better calibration fit to current 

climate conditions can often be obtained using simpler 

temperature-based approaches when the energy inputs are 

subject to uncertainty. 

The calibration objectives for both HSPF and SWAT were 

to achieve error statistics for total streamflow volume, 

seasonal streamflow volume, and high and low streamflow 

within recommended ranges (Donigian 2000, Lumb and 

others 1994) while also maximizing the NSE (Nash and 

Sutcliffe 1970). Because the objectives of this application 

focused at the large basin scale, calibration was undertaken 

only at the HUC 8-digit and larger watershed scale. For 

this application of HSPF, four model parameters were 

the primary focus during model calibration to improve 

model fit for hydrology: INFILT (index to mean soil 

infiltration rate), AGWRC (groundwater recession rate), 

LZSN (lower zone nominal soil moisture storage), and 

BASETP (ET by riparian vegetation). For SWAT, 11 

model parameters were adjusted during model calibration 

to improve model fit in this application: curve number; 

SECO (soil evaporation compensation factor); SURLAG 

(surface runoff lag coefficient); groundwater “revap” rates; 

baseflow factor; GW_DELAY (groundwater delay time); 

GWQMN (threshold depth of water in the shallow aquifer 

required for return flow to occur); RevapMN (threshold 

depth of water in the shallow aquifer required for “revap” 

or percolation to the deep aquifer to occur); CANMAX 

(maximum canopy storage); Manning’s “n” value for 

overland flow, main channels, and tributary channels; and 

Sol_AWC (available water capacity of the soil layer [mm 

water/mm of soil]). 

PRMS-SERAP and PRMS-DAYMET

Two applications of the USGS PRMS were evaluated in 

this study: (1) a PRMS model developed for the USGS 

Southeast Regional Assessment Project (PRMS-SERAP), 

and (2) a PRMS model developed for the USGS National 

Water Census using the DAYMET daily climate input data 

provided by the Oak Ridge National Laboratory (PRMSDAYMET [Daily Meteorological Data]). The PRMS 

(Leavesley and others 1983, Markstrom and others 2008) 

is a deterministic, distributed-parameter, physical-processbased hydrologic modeling system. The model simulates 

daily land-surface hydrologic processes including ET 

by the Jensen-Haise radiation method (Jensen and Haise 

1963), runoff, infiltration, and interflow in HRUs by 

balancing energy and mass budgets of the plant canopy, 

snowpack, and soil zone on the basis of distributed climate 

information (TEMP, PPT, and solar radiation). PRMS 

requires the input of daily maximum and minimum air 

TEMP and daily PPT time-series data. 

PRMS-SERAP—The PRMS-SERAP model was 

developed to provide integrated science that helps 

resource managers understand the effect of climate change 

on a range of ecosystem responses in the ACF Basin 

(LaFontaine and others 2013). For this study, daily climate 

inputs were developed for the PRMS-SERAP model using 

a 1/8-degree gridded TEMP and PPT dataset for 1950–

1999 (Maurer and others 2002). Withdrawals and flow 

regulation were considered during model calibration in 

areas where anthropogenic effects were substantial, mostly 

in the form of irrigation withdrawals (see Wen and Zhang 

2009 for details) and releases from U.S. Army Corps of 

Engineers projects. These withdrawals and regulations 

were then removed in the final model runs to simulate 

unaltered flow volumes.

PRMS-DAYMET—The PRMS-DAYMET model was 

developed to provide flow estimates at several sites 

across the Southeast as part of the USGS National 

Water Census model comparison study. The HRUs 

for the PRMS-DAYMET model were developed by 

aggregating catchments of the medium-resolution National 

Hydrography Dataset (NHDPlus) to an average size of 

78 km2. The daily climate data for PRMS-DAYMET were 

derived from the 1- x 1-km gridded DAYMET daily TEMP 

and PPT dataset for 1980–2008 (Thornton and others 

2013). No anthropogenic withdrawals were included. 

For the PRMS applications in this study, soil-zone, 

subsurface, and groundwater-reservoir parameters were 

computed using the Soil Survey Geographic database 

(SSURGO) (https://www.nrcs.usda.gov/wps/portal/nrcs/

detail/soils/survey/geo/?cid=nrcs142p2_053627), maps 

of near-surface permeability (Gleeson and others 2011), 

and hydrographs of USGS stream gauges. Land cover 

characteristics, such as impervious area, canopy density, 

and land cover type, were obtained from the 2001 NLCD 

(Homer and others 2007) and summarized per HRU. 

For land cover type, each HRU was assigned one of four 
USDA Forest Service, Southern Research Station 15 e-General Technical Report SRS-246

CHAPTER 3

vegetation cover classes (bare soil, grasses, shrubs, trees) 

based on the dominant land cover type, defined as having 

the largest percentage of HRU area. 

An automated parameter estimation procedure (Duan 

and others 1994) was combined with a geographically 

nested approach to calibrate the PRMS models in two 

phases with a total of four steps using the Luca software 

(Hay and Umemoto 2006, Hay and others 2006). The 

first phase of calibration involved matching simulated 

and measured solar radiation (three model parameters) 

and PET (one model parameter), and the second phase of 

calibration involved matching simulated and measured 

streamflow volumes and timing (20 model parameters). 

The objective function in Phase I of model calibration was 

to minimize the absolute difference between predicted and 

observed monthly solar radiation and PET. The objective 

function in Phase II of model calibration depended on the 

application of PRMS. For PRMS-SERAP, the objective 

function in Phase II of model calibration was to minimize 

the normalized root mean squared error (RMSE) of 

flow volumes on an annual, monthly, and mean monthly 

basis and to minimize the normalized RMSE of daily 

flow timing using a 3-day moving average. For PRMSDAYMET, Phase II was expanded to four steps with the 

objective functions targeted at minimizing the normalized 

RMSE at a daily time step instead of the 3-day moving 

average of the PRMS-SERAP model. The four steps of 

calibration for PRMS-DAYMET included (1) matching 

annual, monthly, and mean-monthly flow volumes; 

(2) matching all daily streamflow values; (3) matching 

high-flow days only; and (4) matching low-flow days only.

WATER

WATER, developed within the USGS Center for 

Applied Hydrologic Solutions, is a spatially distributed, 

object-oriented, decision-support system that combines 

the expertise of numerous hydrologists, pedologists, 

computer scientists, and other discipline experts into 

a user-friendly computer application for managing 

water resources. WATER is based on the TOPMODEL 

hydrologic model and incorporates physiographic data 

that quantitatively describe topography and soil water 

storage. TOPMODEL, which simulates the variablesource-area concept of streamflow generation (Wolock 

1993), is well documented (Beven and Kirkby 1979) 

and has been successfully applied in many environments 

(Beaujouan and others 2001, Boyer and others 1996, 

Engel and others 2002). There are three fundamental 

assumptions associated with TOPMODEL: (1) steadystate recharge to the groundwater; (2) hydraulic gradient 

of the water table that approximates the surface slope; and 

(3) a transmissivity profile that decreases exponentially 

with depth. The foundation of TOPMODEL is built on 

the assumption that the land surface is pervious to rainfall 

and that water movement is a function of topography. 

Separate calculations were added from the USDA Natural 

Resources Conservation Service (NRCS) TR-55 method 

(USDA NRCS 1986) to simulate the disposition of water 

for impervious areas of the basin using runoff curve 

numbers specific to pavement. This simulated runoff 

from impervious-surface areas was then input to the 

TOPMODEL mass-balance equation at the beginning of 

each time step to partition runoff derived from pervious 

areas into its surface and subsurface components.

WATER as applied here was developed to provide historic 

and baseline streamflow hydrographs for several USGS 

gauge locations in the ACF Basin based on an approach 

developed for Kentucky (Williamson and others 2009, 

2013) with a combination of the U.S. General Soil Map 

(STATSGO2) and the SSURGO. Land cover data from 

both the 1992 NLCD (Vogelmann and others 2001) and 

2006 NLCD (Fry and others 2011) were used in model 

simulations. Both the National Weather Service River 

Forecast Centers Next-Generation Radar (NEXRAD, 

hereafter NR) and Inverse Distance Weighted (IDW; 

Hay and others 2002) climate inputs were used. Three 

combinations of land cover and climate inputs were 

evaluated in this study and are identified as WATER IDW 

1992, WATER IDW 2006, and WATER NR 2006. 

WATER was calibrated for six basins in the ACF Basin 

using a combination of manual calibration and the 

Parameter Estimation Tool (PEST) (Doherty 2008) on 

daily flows. For PEST, the weight of each observation was 

defined as 

ω = yi× σ2

where 

 w is the weight of an individual observation

 yi

is an individual daily observation

 s2

 is the variability in error for an individual observation 

If yi

< mean daily flow for the site, then w = 0.1; if yi> 

mean daily flow for the site, then w = 0.2.

For days where PPT occurred, w = 0; flow on event days 

was given a zero weight because WATER uses a random 

distribution of daily PPT, so it is unlikely that the initial 

flow from a PPT event will be matched simply due to 

differences in how PPT was modeled versus how the event 

actually occurred. WATER was calibrated by adjusting 

rooting depth and a spatial coefficient that scales the 

topographic wetness index. 
16

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

WaterFALL®

Research Triangle Institute’s WaterFALL® modeling 

system employs an updated version of a well-established 

hydrologic model, the Generalized Water Loading 

Function (GWLF) (Haith and Shoemaker 1987, Haith 

and others 1992), that has been modified to (1) run on 

the U.S. Environmental Protection Agency’s (USEPA) 

enhanced NHDPlus hydrologic network, (2) accept 

parameterization from national datasets, and (3) include 

the impacts of human alterations on streamflows. 

WaterFALL® utilizes the hydrologic simulation within 

GWLF for each catchment of the NHDPlus network in 

the watershed of interest and then accumulates and moves 

water downstream with an embedded time-lagged routing 

routine, providing a distributed hydrologic model across 

the NHDPlus. Parameterization of WaterFALL® occurs 

through the geoprocessing of national datasets for land 

cover, soils, and climate (mean daily TEMP and PPT) 

to each NHDPlus catchment. Each land cover class in a 

catchment is characterized by the predominant HSG and 

predominant percent sand, silt, and clay for the underlying 

soil forming an HRU for runoff calculation within the 

catchment. Mean TEMP and PPT are quantified by 

catchment. Additional basin-specific characteristics such 

as water uses may also be indexed to each catchment from 

local data sources. 

Like SWAT and WATER, surface runoff in WaterFALL®

is computed on a daily basis using the curve number 

method across each land cover type in a catchment 

(USDA Soil Conservation Service 1972). Discharge 

from shallow groundwater is computed using a lumped 

parameter catchment-level water balance for unsaturated 

and shallow saturated zones controlled by the available 

water capacity (AWC) of the unsaturated zone, a recession 

coefficient (RCoeff) providing the rate of release from 

the saturated zone to the stream channel, and a firstorder approximation of infiltration losses to deep aquifer 

storage simulated using a seepage coefficient (SEEP). 

The seepage release constitutes a loss from the system, 

where the water is no longer available to reach the stream 

in the temporal context of daily rainfall-runoff modeling. 

Daily ET from the unsaturated zone is computed using a 

land use-based cover factor, and PET is computed using 

the Hamon temperature-based method (Hamon 1963). 

Three model parameters (AWC, RCoeff, and SEEP) are 

adjusted during an automated calibration process using a 

customized version of the PEST. Because of the physical 

basis of the AWC and RCoeff parameters, a priori values 

for the parameters are indexed to individual catchments 

within the WaterFALL® database, and a multiplier across 

the physically based values is adjusted during calibration. 

The SEEP parameter is set through calibration and 

consideration of local conditions.

The application of WaterFALL® used for this study was 

developed to create a hydrologic foundation for detailed 

assessment of human and climate effects on stream and 

river flows, including the effects of hydrologic alterations 

on aquatic habitats in the SEUS at the NHDPlus catchment 

scale (~1.0 km2) (Kendy and others 2011). For this study, 

climate inputs included daily PPT and TEMP from a 

4- x 4-km national dataset obtained from the USDA 

(Di Luzio and others 2008), land cover was based on 

the baseline condition assessment represented by the 

USGS Geographic Information Retrieval and Analysis 

System (GIRAS) land cover (Price and others 2006), 

and soils data were obtained from SSURGO (https://

www.nrcs.usda.gov/wps/portal/nrcs/detail/soils/survey/

geo/?cid=nrcs142p2_053627). Although water system 

discharges and withdrawals were obtained from State 

databases on public and non-public systems for other 

scenarios in the SEUS study, the WaterFALL® model 

simulations contributing to this study were unaltered by 

humans. WaterFALL® was calibrated at several USGS 

gauge locations in the study area for periods in the 1970s, 

commensurate with the land use coverage used to simulate 

a less-altered baseline for the SEUS study. Although 

performance was validated with later periods, some 

differentiation from the other models is expected where 

the calibration period more closely matched the period of 

this study. The three model calibration parameters (AWC, 

RCoeff, and SEEP) were optimized to minimize the 

differences in log-transformed daily flows, giving equal 

weight to differences in streamflows at the low and high 

end of the hydrograph. 

MWBM

The USGS MWBM (Hay and McCabe 2002, McCabe 

and Markstrom 2007, McCabe and Wolock 2011) is 

based on the monthly Thornthwaite water balance model 

(Thornthwaite 1948). PET is calculated from monthly 

TEMP using the Hamon equation (Hamon 1963). When 

PPT exceeds PET in a given month, actual ET is equal to 

PET. Water in excess of PET replenishes soil-moisture 

storage. When soil-moisture storage reaches field capacity 

during a given month, the excess water becomes surplus. 

In a given month, some percentage of the total surplus 

becomes runoff, and the remaining surplus is carried 

over to the following month. Like PRMS-DAYMET, the 

MWBM used in this study was developed to provide flow 

estimates at several sites across the Southeast as part of the 

USGS National Water Census model comparison study. 

The model was run on the same HRUs as were developed 

for the PRMS-DAYMET model and used PRISM 

(Parameter-elevation Relationships on Independent Slopes 

Model)-based monthly precipitation and TEMP estimates 

(PRISM Climate Group 2013). Four model parameters 

were adjusted during model calibration: PETfac (a PET 
USDA Forest Service, Southern Research Station 17 e-General Technical Report SRS-246

CHAPTER 3

correction factor), ROfac (percentage of the total surplus 

that becomes runoff), WHC (water holding capacity), 

and Pbias (monthly PPT bias). The calibration objective 

was to minimize normalized RMSE and bias using the 

same algorithms as the PRMS-SERAP and PRMSDAYMET models.

WaSSI

The WaSSI model was developed by the Forest Service 

to assess the effect of climate change, land use change, 

and population growth on water supply stress, river flows, 

and aquatic ecosystems across the conterminous United 

States (Caldwell and others 2012, Sun and others 2011b). 

WaSSI has been successfully used in climate change 

assessments in the Eastern United States (Lockaby and 

others 2011, Marion and others 2013, Sun and others 

2013, Tavernia and others 2013) and for examining the 

nexus of water and energy at the national scale (Averyt 

and others 2011, 2013). WaSSI is an integrated monthly 

water balance and flow routing model that simulates the 

full hydrologic cycle for each of 10 land cover classes at 

the 12-digit Hydrologic Unit Code (HUC) scale. The 10 

land cover classes are aggregated from the 17 classes of 

the 2006 National Land Cover Database (NLCD) (Fry and 

others 2011). Infiltration, surface runoff, soil moisture, and 

baseflow processes for each HUC watershed land cover 

were computed using algorithms of the Sacramento Soil 

Moisture Accounting Model (SAC-SMA) (Burnash 1995, 

Burnash and others 1973). STATSGO databases (USDA 

NRCS 2012) were used to compute the 11 SAC-SMA 

soil input parameters (Koren and others 2003). Monthly 

ET was modeled with an empirical equation derived 

from multisite eddy covariance ET measurements (Sun 

and others 2011a, 2011b). Required data to estimate ET 

included monthly mean Moderate Resolution Imaging 

Spectroradiometer (MODIS) MOD15A2 leaf area index 

(LAI) (Zhao and others 2005), Hamon PET calculated as 

a function of TEMP and latitude (Hamon 1963), and PPT. 

This estimate of ET was then constrained by the soil water 

content computed by the SAC-SMA algorithm during 

extreme water-limited conditions. Monthly PPT and air 

TEMP inputs were based on PRISM estimates (PRISM 

Climate Group 2013). All water balance components were 

computed independently for each land cover class within 

each HUC watershed and accumulated to estimate the 

totals for the watershed. For the NLCD-based impervious 

cover fraction, storage and ET were assumed to be 

negligible, and thus all PPT falling on the impervious 

portion of a watershed for a given month was assumed to 

generate surface runoff in the same month and was routed 

directly to the watershed outlet. No anthropogenic water 

use was included, and the model was run using off-theshelf input datasets without calibration. 

Site Description and Observed Streamflow Sites

The study area included USGS streamflow gauges across 

eight States in the SEUS (fig. 3.2). Central to the study 

area is the ACF Basin which drains approximately 52,000 

km2 of Georgia, Alabama, and Florida. For over 2 decades, 

there have been periodic conflicts over water resources 

in the ACF Basin among Georgia, Alabama, and Florida 

and other stakeholders that depend on the river system for 

public supply, industry, power generation, and agriculture. 

As the region has grown and developed over the past 50 

years, competition among all water users has become 

more pronounced, particularly during drought conditions 

(Seager and others 2009). As a result, the basin has been 

intensely studied over the last decade with multiple 

modeling efforts taking place to evaluate drought and 

environmental change effects on water supply and aquatic 

ecosystems (e.g., Alley and others 2013, Freeman and 

others 2013, Georgakakos and others 2010, LaFontaine 

and others 2013).

The gauges were selected because they had long records 

of continuous streamflow measurements, and, to our 

knowledge, there were no major dams, diversions, etc. that 

substantially altered the flow regime. While we attempted 

to limit this study to basins with minimally altered flow 

regimes, there could be unknown anthropogenic processes 

in the basins that are not accounted for in the models and 

could affect model predictive performance. Drainage 

areas of the 195 sites ranged from 14 to 44 548 km2. The 

number of gauges and temporal extents varied among 

models (fig. 3.2, table 3.3), so model performance was 

not directly comparable across all gauges. However, a 

subset of gauges (table 3.4) were simulated by the HSPF, 

PRMS-SERAP, PRMS-DAYMET, SWAT, WaterFALL®, 

MWBM, and WaSSI models (i.e., all models except the 

three parameterizations of WATER) from 1980–1999. 

Models were compared in two ways: first, by comparing 

all models across all gauges and time periods simulated 

irrespective of whether or not simulations were performed 

at the same sites (fig. 3.2, table 3.3), and second, by 

directly comparing model predictions at the subset of 

gauges simulated by HSPF, PRMS-SERAP, PRMSDAYMET, SWAT, WaterFALL®, MWBM, and WaSSI 

models (fig. 3.3, table 3.4).

The five gauges simulated by HSPF, PRMS-SERAP, 

PRMS-DAYMET, SWAT, WaterFALL®, MWBM, and 

WaSSI models from 1980–1999 were all located in the 

ACF Basin (fig. 3.3) ranging in drainage area from 637 to 

4792 km2

 with mean annual PPT ranging from 1282 mm 

to 1728 mm and mean annual TEMP ranging from 13.9 

to 18.7 °C (table 3.4). Land cover ranged from mostly 

forested (site 1: Chattahoochee River near Cornelia, GA), 
Gauges

Number of models

1

2

3

4

5

6

7

10

ACF Basin

HUC-6 watersheds

 ̄

0 200100 km

18

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Figure 3.2—Spatial distribution of USGS streamflow gauges in the SEUS used to evaluate model 

performance in predicting observed streamflows. Colors of the gauge sites indicate the number of 

models that simulated streamflow for that site.

to highly urbanized (site 2: Sweetwater Creek near Austell, 

GA), to mixed agriculture and forest along the southern 

extent of the ACF Basin (site 5: Ichawaynochaway Creek 

at Milford, GA). The amount of impervious area ranged 

from 0.4 to 9.4 percent and was highly related to the 

amount of developed land in the basin (R2 = 0.986).

Evaluation and Comparison of Error 

in Modeled Flows

In this study, we compared classical hydrologic model fit 

statistics among model applications, sites, and levels of 

calibration (1) over all sites and time periods simulated by 

each model and (2) over the five sites simulated by HSPF, 

PRMS-SERAP, PRMS-DAYMET, SWAT, WaterFALL®, 

MWBM, and WaSSI models from 1980–1999. Classical 

hydrologic model fit statistics were computed for each 

model and site at the monthly time step for all seven 

models and at the daily time step for hourly and daily 

models only. Fit statistics evaluated included RMSE, R2, 

bias in mean streamflow, and the NSE statistic (Nash and 

Sutcliffe 1970). The NSE can range from negative infinity 

to 1.0; the closer NSE is to 1.0, the better the model fit to 

observations. Negative values of NSE indicate that using 

the mean of the observations provides a better fit than 

the model. NSE values that are >0.50, >0.65, and >0.75 

for prediction of monthly streamflow have been viewed 

as indicative of satisfactory, good, and very good model 

performance, respectively (Moriasi and others 2007). 

Similarly, bias in mean streamflow within ±25, ±15, and 

±10 percent is indicative of satisfactory, good, and very 

good model performance, respectively (Moriasi and 

others 2007).

As discussed previously, models evaluated in this study 

were developed by different agencies, for different 

purposes, with different data inputs and spatial scales, and 

were calibrated using different objective functions. Some 

models were not calibrated at all, and some calibrated 

models were either specifically calibrated at these sites 

during model development or were calibrated for another 

streamflow gauge downstream. Some model calibration 

included adjusting PPT, solar radiation, and PET inputs. 

To examine the role that the level of model calibration 

plays in predictive performance, we define four levels 

of increasing calibration intensity to form a basis for 

comparison among models: calibration level A models 

are uncalibrated (i.e., WaSSI); calibration level B models 

are calibrated to a downstream gauge (i.e., some HSPF, 

SWAT, and WaterFALL® sites); calibration level C models 

are calibrated specifically for that site (i.e., some HSPF, 

SWAT, WATER, and WaterFALL® sites); and calibration 

level D models are calibrated specifically for that site, 
USDA Forest Service, Southern Research Station 19 e-General Technical Report SRS-246

CHAPTER 3

Table 3.3—Summary of calibration level for gauges simulated by each model 

Model

Number of 

gauges

Drainage area 

(km2)

Time period

Number of gauges in 

calibration levela

Minimum Maximum Median A B C D

HSPF 12 2244 44 548 4274 10/1973–9/2003 0 3 9 0

PRMS-SERAP 27 92 44 548 834 10/1951–12/1999 0 0 0 27

PRMS-DAYMET 174 14 39 075 435 10/1980–9/2011 0 0 0 174

SWAT 12 2244 44 548 4274 10/1973–9/2003 0 3 9 0

WATER IDW 1992 3 262 4792 3160 1/1981–12/2008 0 0 3 0

WATER IDW 2006 9 23 7563 705 1/1981–12/2008 0 0 9 0

WATER NR 2006 9 23 7563 521 10/2001–12/2010 0 0 9 0

WaterFALL® 26 14 4792 254 1/1980–12/2006 0 2 24 0

MWBM 175 14 39 075 440 1/1980–12/2011 0 0 0 175

WaSSI 184 14 44 548 456 10/1980–9/2010 184 0 0 0

a Calibration levels include: (A) uncalibrated, (B) calibrated to downstream gauge, (C) calibrated specifically for site, and (D) calibrated specifically for 

site with adjusted precipitation, solar radiation, and potential evapotranspiration (PET) inputs. 

DAYMET = Daily Meteorological Data; HSPF = Hydrological Simulation Program-Fortran; IDW = inverse distance weighted; MWBM = Monthly Water 

Balance Model; NR = next generation radar; PRMS = Precipitation-Runoff Modeling System; SERAP = Southeast Regional Assessment Project; 

SWAT = Soil and Water Assessment Tool; WaSSI = Water Supply Stress Index model; WATER = Water Availability Tool for Environmental Resources; 

WaterFALL® = Watershed Flow and Allocation modeling system using NHDPlus.

Table 3.4—Descriptionsa of the five U.S. Geological Survey continuous record gauges simulated by HSPF, PRMSSERAP, PRMS-DAYMET, SWAT, WaterFALL®, MWBM, and WaSSI models (1980–1999)

Site number 1 2 3 4 5

Station ID 02331600 02337000 02342500 02347500 02353500

Station name Chattahoochee River 

near Cornelia, GA

Sweetwater Creek 

near Austell, GA

Uchee Creek near 

Fort Mitchell, AL

Flint River at 

US 19, near 

Carsonville, GA

Ichawaynochaway 

Creek at Milford, GA

Drainage area (km2

) 816 637 834 4792 1606

Screening comments 

 for alteration

Forested headwaters, 

mixed light agriculture/

forest/suburban, many 

flood retention ponds 

on tributaries

Urban basin Urban in 

headwaters of 

main tributary

Upstream urban Proximate irrigated 

agriculture, many small 

ponds in headwaters, 

small towns on some 

tributaries

Developed 8.8% 39% 8.1% 13.9% 3.3%

Impervious 1.1% 9.4% 1.3% 3.6% 0.4%

Forested 72% 41% 52% 55% 39%

Agriculture 13% 11% 16% 16% 37%

Mean TEMP (°C) 13.9 15.3 17.4 16.8 18.7

Mean PPT (mm yr-1) 1728 1384 1320 1282 1330

Mean runoff (mm yr-1) 615 497 448 411 426

Runoff coefficient 36% 36% 34% 32% 32%

a

 Descriptive watershed data is from Falcone and others (2010) and Falcone (2011).

PPT = precipitation; TEMP = air temperature.
#

#

#

#

#

!

!

! !

!

! !

!

!

!

!

!

! !

!

Tampa

Athens

Atlanta Augusta

Columbus Savannah

Clearwater

Montgomery

Birmingham

Huntsville

Gainesville

Tallahassee

Jacksonville

GA

FL

AL

SC

TN NC

 Legend

# Gauge station

Gauged basins

Apalachicola-Chattahoochee-Flint Basin

 ̄ 0 10050 km

1

2

4

5

3

20

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Figure 3.3—Map showing the location of the five USGS continuous record 

gauges and associated drainage basins in the Apalachicola-ChattahoocheeFlint Basin used to compare simulated flow predictions from the HSPF, 

PRMS-SERAP, PRMS-DAYMET, SWAT, WaterFALL®, MWBM, and WaSSI models 

(1980–1999).

and PPT, solar radiation, and PET inputs were adjusted as 

part of the calibration process to account for uncertainty 

in gridded climate estimates (i.e., PRMS models and 

MWBM) (tables 3.3 and 3.5).

We also evaluated differences between predicted and 

observed ERFMs across the five hourly and daily models 

over the five sites simulated by HSPF, PRMS-SERAP, 

PRMS-DAYMET, SWAT, and WaterFALL® models from 

1980–1999. The monthly MWBM and WaSSI models 

were not evaluated for prediction of ERFMs because many 

of the ERFMs require a daily time step to be calculated. 

Predicted and observed daily mean flows were imported 

into the EflowStats package, an R version of the National 

Hydrologic Assessment Tool (NATHAT) (Henriksen 

and others 2006) developed by the USGS Center for 

Integrated Data Analytics and available on GitHub at 

https://github.com/USGS-R/EflowStats. This R package 

was developed to assist water resource professionals with 

characterizing the five major components of the flow 

regime (i.e., magnitude, frequency, duration, timing, and 

rate of change) considered by many to be important in 

shaping ecological processes in streams (Henriksen and 

others 2006, Kennen and others 2007, Olden and Poff 

2003). A total of 175 ERFMs were evaluated for this 

study. Scatterplots were used to examine data distributions 

and to detect potential outliers in the ERFMs; metrics 

with extreme outliers or with highly limited data ranges 

were removed from further consideration. A Spearman 

rank correlation matrix (SAS Institute Inc. 1989) on the 

reduced set of ERFMs was then examined to eliminate 

any remaining redundant variables with a Spearman’s 

rho >0.75. In cases where two metrics accounting for 

similar aspects of the flow regime were highly collinear, 

selection was based on best professional judgment. 

This approach was highly parsimonious and permitted 
USDA Forest Service, Southern Research Station 21 e-General Technical Report SRS-246

CHAPTER 3

Table 3.5—Level of model calibration by site for the five sites simulated by HSPF, PRMSSERAP, PRMS-DAYMET, SWAT, WaterFALL®, MWBM, and WaSSI models from 1980–1999 

Model

Number of 

parameters 

adjusted Calibration objective function Site

Level of 

calibrationa

HSPF 4 Total, seasonal, high, and low streamflow 

within recommended ranges (Donigian 2000, 

Lumb and others 1994), maximize daily NSE

1 C

2 B

3 B

4 B

5 C

PRMS-SERAP 24 Minimize normalized RMSE of annual, monthly, 

and mean monthly flow volumes, and minimize 

normalized RMSE of daily flow timing using a 

3-day moving average

1 D

2 D

3 D

4 Db

5 D

PRMS-DAYMET 24 Minimize normalized RMSE of annual, monthly, 

mean monthly, and daily flow volumes

1 D

2 D

3 D

4 D

5 D

SWAT 11 Total, seasonal, high, and low streamflow 

within recommended ranges (Donigian 2000, 

Lumb and others 1994), maximize daily NSE

1 C

2 B

3 B

4 B

5 C

WaterFALL® 3 Minimize bias in log-transformed daily flows 1 C

2 C

3 B

4 C

5 B

MWBM 4 Minimize normalized RMSE of annual, monthly, 

and mean monthly flow volumes

1 D

2 D

3 D

4 D

5 D

WaSSI 0 N/A 1 A

2 A

3 A

4 A

5 A

a

 Calibration levels include: (A) uncalibrated, (B) calibrated to downstream gauge, (C) calibrated specifically for site, 

and (D) calibrated specifically for site with adjusted precipitation, solar radiation, and potential evapotranspiration 

(PET) inputs.

b

 PRMS-SERAP was calibrated to a downstream gauge at site 4, but this calibration adjusted precipitation, solar 

radiation, and potential evapotranspiration inputs. For the purpose of comparison among calibration levels, PRMSSERAP for this site was set at level D.

DAYMET = Daily Meteorological Data; HSPF = Hydrological Simulation Program-Fortran; MWBM = Monthly Water 

Balance Model; NSE = Nash-Sutcliffe Efficiency; PRMS = Precipitation-Runoff Modeling System; RMSE = root mean 

squared error; SERAP = Southeast Regional Assessment Project; SWAT = Soil and Water Assessment Tool; WaSSI = 

Water Supply Stress Index model; WaterFALL® = Watershed Flow and Allocation modeling system using NHDPlus.
22

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

the retention of important streamflow metrics which 

were highly interpretable and management oriented. 

It also helped avoid the possibility of establishing 

significant (p <0.05) correlations among a large suite of 

hydrologic variables simply by chance and introducing 

interdependencies among multiple explanatory variables 

(King and others 2005, Van Sickle 2003). Bias in the 

resulting subset of ERFMs was quantified by computing 

the percent difference between the predicted and observed 

flow metric for each model and site. A hydrologic 

uncertainty of ±30 percent (hereafter range of uncertainty) 

was used to aid in placing model prediction bias into 

context with inherent variability in streamflow and flow 

measurement (Murphy and others 2013). 

All model streamflow predictions were compared to 

USGS flow observations at each site. After comparing 

classical fit statistics and prediction of ERFMs for each 

model, we explored differences in model structure, 

inputs, and calibration strategy that may explain 

differences in predictive performance. In addition, we 

examined differences in model input for monthly PPT 

and predicted ET, runoff, and soil moisture at site 4 

(02347500, Flint River near Carsonville, GA; table 3.4; 

fig. 3.3) and the role these differences may play in the 

predicted water balance and model error for the HSPF, 

PRMS-SERAP, PRMS-DAYMET, SWAT, WaterFALL®, 

MWBM, and WaSSI models. PPT, ET, and runoff were 

directly comparable among all five models; however, soil 

moisture is represented differently in the flow models. 

To accommodate direct comparison, we standardized 

soil moisture predictions by dividing the monthly mean 

soil moisture by the maximum soil moisture storage for 

each model.

RESULTS

All Gauges Simulated by Each Model

Median model fit statistics over all gauges simulated by 

each model are shown in table 3.6. Note that the model fit 

statistics in table 3.6 are not directly comparable across 

models because the specific gauges and time periods 

simulated differed among models. In general, models 

captured the magnitude, variability of observed flows, and 

mean flow for the gauges and time periods they simulated 

with median absolute bias in mean flow <6 percent across 

most models. The WATER IDW simulations tended to 

show high positive bias in many of the fit statistics. There 

was a tendency among some models to overestimate low 

flows. For example, the median bias in 10th percentile 

monthly flows was +17.1, +7.6, +37.0, +4.8, and +15.0 

percent for HSPF, PRMS-SERAP, PRMS-DAYMET, 

WaterFALL®, and WaSSI, respectively. High flows 

were generally well predicted, with median bias in 90th

percentile monthly flows within ±4.3 percent among most 

models. Although bias in predicted low flows was greater 

than that of high flows in relative terms (i.e., percent), 

there was little difference in low- and high-flow bias in 

absolute terms (i.e., cubic feet per second [cfs]). With the 

exception of WATER NR 2006, models generally underpredicted the coefficient of variation (CV) of observed 

flows regardless of the time scale. For example, the 

median bias in the CV in daily flows among daily time 

step models ranged from -2.6 percent (WaterFALL®) to 

-20.8 percent (PRMS-DAYMET), while that of WATER 

NR 2006 was +6.2 percent. Winter (December–February) 

and early spring (March–May) flows were generally well 

predicted, but late spring, summer, and early fall flows 

(June–November) were often over-predicted. This likely 

reflects the over-prediction of low flows because the 

lowest flows on an annual basis typically occur during the 

summer and early fall months. Model performance was 

satisfactory or better according to the Moriasi and others 

(2007) criteria for monthly NSE (>0.50) and bias in mean 

flow (within ±25 percent) at most of the gauges simulated 

by each model (fig. 3.4, table 3.7). 

Comparison of Models for the Five 

Gauges Simulated by Select Models 

from 1980–1999

As mentioned previously, the number of gauges and 

temporal extents varied among models, so model 

performance was not directly comparable across all 

gauges. However, a subset of gauges (table 3.4) were 

simulated by the HSPF, PRMS-SERAP, PRMS-DAYMET, 

SWAT, WaterFALL®, MWBM, and WaSSI models (i.e., 

all models except the three parameterizations of WATER) 

from 1980–1999, allowing for direct comparisons among 

these models for the same sites and time periods. A 

complete summary of this comparison can be found in 

Caldwell and others (2015). In general, HSPF, PRMSSERAP, PRMS-DAYMET, SWAT, WaterFALL®, MWBM, 

and WaSSI models captured the magnitude and variability 

of observed streamflows at the five study sites (table 3.8, 

figs. 3.5 and 3.6). The median bias in mean streamflow 

across sites by model ranged from -15.1 percent 

(WaterFALL®, site 5, calibration level B) to +15.4 percent 

(WaSSI, site 5, calibration level A), while median absolute 

bias across sites by model ranged from 2.5 percent 

(PRMS-DAYMET, site 1, calibration level D) to 15.4 

percent (WaSSI, site 5, calibration level A). The median 

monthly NSE across all five study sites by model ranged 

from 0.64 (SWAT, site 2, calibration level B) to 0.87 

(PRMS-SERAP, site 4, calibration level D). As expected, 

fit statistics for the daily models at the monthly time step 

were superior to the fit statistics at the daily time step for 

daily and sub-daily models. The median daily NSE across 

sites for the daily models (table 3.8) ranged from 0.37 

(HSPF, site 5, calibration level C) to 0.80 (PRMS-SERAP, 

site 5, calibration level D). Using model performance 

criteria for bias and monthly NSE (Moriasi and others 
USDA Forest Service, Southern Research Station 23 e-General Technical Report SRS-246

CHAPTER 3

Table 3.6—Median model fit statistics over all gauges simulated by each model 

Statistica HSPF

PRMSSERAPPRMSDAYMET SWAT

WATER 

IDW 

1992

WATER 

IDW 

2006

WATER 

NR 

2006 WaterFALL® MWBM WaSSI

Number of gauges 12 27 174 12 3 9 8 26 175 184

Calibration levelb B, C D D B, C C C C B, C D A

Bias mean flow 5.9% 2.8% 3.8% 3.2% 38.9% 32.4% -0.3% 2.6% -0.9% 5.7%

Bias median daily flow 3.5% 10.7% 22.0% 6.1% 37.2% 9.1% -41.3% 4.1% N/A N/A

Bias median monthly flow 5.5% 4.5% 10.6% -0.0% 19.5% 4.0% -43.5% 6.6% -0.8% 14.9%

Bias median annual flow 2.1% 4.0% 4.4% 1.0% 35.5% 35.8% 4.2% 1.5% -1.1% 7.1%

Bias 10th percentile daily flow 9.6% -7.0% 25.4% -18.5% 4.4% -7.8% -25.7% 2.7% N/A N/A

Bias 10th percentile monthly flow 17.1% 7.6% 37.0% -9.3% -11.2% -22.0% -32.5% 4.8% -7.6% 15.0%

Bias 90th percentile daily flow 1.4% -1.0% 1.4% -1.6% 60.2% 57.3% 23.3% 9.6% N/A N/A

Bias 90th percentile monthly flow 4.3% -0.8% -3.2% 0.5% 66.7% 51.4% 32.0% -2.8% -0.6% -0.1%

Bias CV daily flow -4.1% -16.0% -20.8% -18.6% -13.6% -15.4% 6.2% -2.6% N/A N/A

Bias CV monthly mean flow 2.3% -4.0% -13.8% 9.2% 5.7% 0.6% 26.5% -10.5% -4.9% -10.7%

Bias CV annual mean flow -12.7% -3.5% -12.3% 8.0% 7.8% 1.6% 25.0% -11.0% -0.9% -7.9%

Bias January mean flow 6.0% 0.1% 8.4% 2.0% 58.6% 53.9% -4.5% 9.5% -0.6% 9.0%

Bias February mean flow 6.5% -1.7% -0.8% 4.2% 57.9% 66.4% 19.4% 2.3% 3.0% 1.5%

Bias March mean flow 4.3% -5.2% -11.6% 3.5% 50.7% 60.0% 26.2% -5.5% -2.9% -8.4%

Bias April mean flow 5.3% -4.6% -12.4% 4.8% 51.6% 49.3% 31.0% -7.6% -0.3% -7.4%

Bias May mean flow 11.6% 1.8% -5.9% 11.1% 33.0% 10.3% -1.4% 0.0% -0.0% -5.4%

Bias June mean flow 21.7% 18.7% 7.4% 12.6% 9.4% -15.0% -32.7% 0.1% 9.3% 10.7%

Bias July mean flow 12.3% 16.7% 15.1% 8.4% -9.3% -17.2% -28.3% 0.9% 2.1% 17.3%

Bias August mean flow 12.9% 26.9% 27.0% -3.4% 20.0% -1.4% -42.7% 4.1% -4.7% 27.9%

Bias September mean flow 9.3% 26.0% 30.0% 2.7% 20.9% -20.6% -35.6% 7.2% -4.6% 31.5%

Bias October mean flow 8.0% 10.1% 26.5% 2.7% 17.4% 7.1% -7.8% 15.0% -12.5% 22.1%

Bias November mean flow 1.7% -4.6% 15.5% -5.1% 24.2% 22.1% -4.9% 6.1% 0.4% 19.9%

Bias December mean flow -1.8% -7.5% 11.6% -9.0% 54.3% 56.9% 2.4% 8.9% -5.6% 10.4%

NSE daily flow 0.62 0.73 0.69 0.42 0.19 0.13 -0.09 0.30 N/A N/A

NSE monthly flow 0.83 0.83 0.84 0.59 0.30 0.30 0.65 0.79 0.84 0.74

NSE annual flow 0.76 0.80 0.81 0.49 -0.72 -0.47 -0.70 0.66 0.87 0.74

a

 Note that these statistics are not directly comparable across models because they simulated different gauges over different time periods ranging from 

4 to 48 years between 1951 and 2011. 

b

 Calibration levels include: (A) uncalibrated, (B) calibrated to downstream gauge, (C) calibrated specifically for site, and (D) calibrated specifically for site 

with adjusted precipitation, solar radiation, and potential evapotranspiration (PET) inputs.

CV = coefficient of variation; DAYMET = Daily Meteorological Data; HSPF = Hydrological Simulation Program-Fortran; IDW = Inverse Distance Weighted; 

MWBM = Monthly Water Balance Model; NR = Next-Generation Radar; NSE = Nash-Sutcliffe Efficiency; PRMS = Precipitation-Runoff Modeling System; 

SERAP = Southeast Regional Assessment Project; SWAT = Soil and Water Assessment Tool; WaSSI = Water Supply Stress Index model; WATER = Water 

Availability Tool for Environmental Resources; WaterFALL® = Watershed Flow and Allocation modeling system using NHDPlus.
0

10

20

30

40

50

60

70

80

90

100

Absolute bias (%)

Sites with bias equaled or less (%)

HSPF (n = 12)

PRMS-SERAP (n = 27)

PRMS-DAYMET (n = 174)

SWAT (n = 12)

WATER IDW 1992 (n = 3)

WATER IDW 2006 (n = 9)

WATER NR 2006 (n = 8)

WaterFALL® (n = 26)

MWBM (n = 175)

0 20 40 60 80 100 WaSSI (n = 184)

(A)

0

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.0

Monthly NSE

Sites with NSE equaled or greater (%)

0 20 40 60 80 100

(B)

0

10

20

30

40

50

60

70

80

90

100

Absolute bias (%)

Sites with bias equaled or less (%)

HSPF (n = 12)

PRMS-SERAP (n = 27)

PRMS-DAYMET (n = 174)

SWAT (n = 12)

WATER IDW 1992 (n = 3)

WATER IDW 2006 (n = 9)

WATER NR 2006 (n = 8)

WaterFALL® (n = 26)

MWBM (n = 175)

0 20 40 60 80 100 WaSSI (n = 184)

(A)

24

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Figure 3.4—Distribution of bias in mean flow (A) and monthly NSE (B) across all sites simulated by each model. Only 

NSE values >0 are shown. The region highlighted in green is the bound for satisfactory model performance (Moriasi and 

others 2007) with monthly NSE >0.50 and bias in mean flow <±25 percent.

2007), all models had satisfactory or better performance at 

most sites (fig. 3.6). Model performance was satisfactory 

or better for mean streamflow bias (within ±25 percent of 

observed) with the exception of one site for SWAT (site 

2, bias -26.2 percent, calibration level B), two sites for 

WaterFALL® (site 2, bias -33.7 percent, calibration level 

C; site 3, bias -48.0 percent, calibration level B), and one 

site for WaSSI (site 2, bias +25.0 percent, calibration level 

A). Similarly, model performance was satisfactory or 

better for monthly NSE (>0.50) with the exception of one 

site for SWAT (site 5, NSE 0.46, calibration level C) and 

one site for WaterFALL® (site 3, NSE 0.28, calibration 

level B). The median bias in mean streamflow across 

all models by site ranged from -12.0 percent at site 3 to 

+9.2 percent at site 4, while the median monthly NSE 

across models ranged from 0.72 at site 2 to 0.89 at site 1 

(Caldwell and others 2015).

Differences in mean fit statistics across models by site 

were not significant (p >0.05). The mean bias in mean 

flow across models by site ranged from -9.8 percent at site 

1 to +7.2 percent at site 4, while the mean monthly NSE 

ranged from 0.47 at site 3 to 0.86 at site 4. The range in 

fit statistics among models for site 3 was greater than the 

other sites, with HSPF, SWAT, and WaterFALL® occurring 

as outliers compared to the other models and having 

considerably greater bias and lower NSE (table 3.8, fig. 

3.5). The lower level of fit of these models for this site 

relative to the other models is likely because they were 

not calibrated for this site; rather, they were calibrated at a 

downstream gauge. 

Increasing calibration intensity tended to improve model 

fit across sites and models (fig. 3.7). Calibration level 

A (uncalibrated) included only WaSSI simulations, 
USDA Forest Service, Southern Research Station 25 e-General Technical Report SRS-246

CHAPTER 3

Table 3.7—Percent of gauges simulated by each model that could be considered satisfactory, good, and very 

good performance by monthly Nash-Sutcliffe Efficiency (NSE) and bias in mean flow criteria established by 

Moriasi and others (2007) 

Monthly NSE Bias mean flow

Model

Number of 

gauges

Satisfactory

(>0.50)

Good

(>0.65)

Very good

(>0.75)

Satisfactory

(within ±25%)

Good

(within ±15%)

Very good

(within ±10%)

HSPF 12 100% 83% 67% 100% 83% 58%

PRMS-SERAP 27 96% 96% 89% 100% 100% 89%

PRMS-DAYMET 174 98% 96% 89% 98% 93% 80%

SWAT 12 67% 42% 33% 75% 67% 42%

WATER IDW 1992 3 0% 0% 0% 0% 0% 0%

WATER IDW 2006 9 11% 0% 0% 33% 22% 0%

WATER NR 2006 8 88% 50% 13% 100% 100% 75%

WaterFALL® 26 96% 77% 54% 81% 69% 46%

MWBM 175 99% 98% 86% 99% 99% 99%

WaSSI 184 93% 76% 45% 86% 70% 56%

DAYMET = Daily Meteorological Data; IDW = Inverse Distance Weighted; MWBM = Monthly Water Balance Model; NR = Next-Generation Radar; 

NSE = Nash-Sutcliffe Efficiency; PRMS = Precipitation-Runoff Modeling System; SERAP = Southeast Regional Assessment Project; SWAT = Soil and 

Water Assessment Tool; WaSSI = Water Supply Stress Index model; WATER = Water Availability Tool for Environmental Resources; WaterFALL® = 

Watershed Flow and Allocation modeling system using NHDPlus.

calibration level B (calibrated to downstream gauge) 

included three sites for HSPF and SWAT and two sites for 

WaterFALL®, calibration level C (calibrated specifically 

for site) included two sites for HSPF and SWAT and three 

sites for WaterFALL®, and calibration level D (calibrated 

for site with adjusted PPT, solar radiation, and PET 

inputs) included all PRMS-SERAP, PRMS-DAYMET, 

and MWBM simulations (table 3.5). The median absolute 

bias in streamflow across models and sites for a given 

level of calibration decreased from 15 and 17 percent 

for calibration levels A and B, respectively, to 6 percent 

for level C, to 3 percent for level D. Similarly, the 

monthly NSE tended to increase with increasing level of 

calibration. Median monthly NSE across sites and models 

for calibration levels A, B, C, and D, were 0.72, 0.68, 0.75, 

0.85, respectively. Daily NSE across sites and daily timestep models also improved with increasing calibration (not 

shown), with median values of 0.39, 0.37, and 0.74 for 

calibration levels B, C, and D, respectively.

Evaluation of the water balance components at site 4 

revealed the effect of modeling assumptions and 

calibration strategies on model fit statistics. Models 

generally over-predicted streamflow at site 4 (fig. 3.5), and 

thus calibration strategies for some models were aimed 

at either adjusting PPT (i.e., PRMS-SERAP, PRMSDAYMET, and MWBM) or increasing losses through 

deep seepage (i.e., WaterFALL®) (table 3.5). Absolute 

streamflow over-prediction was most prevalent during 

the seasonally high-flow months of January, February, 

and March, but streamflow over-predictions expressed 

as a percentage were highest in the low-flow months of 

July, August, and September (fig. 3.8B). Input for mean 

annual PPT was similar for HSPF, SWAT, WaterFALL®, 

and WaSSI, ranging from 1221 mm (WaterFALL®) to 

1258 mm (HSPF and SWAT) for a total difference of 

approximately 3 percent (fig. 3.8A). Mean annual PPT 

for PRMS-SERAP, PRMS-DAYMET, and MWBM was 

reduced for site 4 during model calibration, resulting in 

decreases in mean annual PPT of approximately 9, 15, 

and 10 percent, respectively, whereas the other models 

did not adjust input PPT. WaterFALL® included deep 

seepage losses (approximately 127 mm, or 10 percent of 

PPT) to reduce streamflow predictions to more closely 

match observations by adjusting the seepage coefficients 

in the calibration process. Had deep seepage not been 

included, bias in mean streamflow for WaterFALL® may 

have increased from 11.3 to 44.2 percent, although other 

model parameters would have likely been adjusted to 

improve model fit. HSPF and SWAT simulations included 

consumptive use terms that also reduced streamflow 

predictions, amounting to approximately 33 mm and 

67 mm, respectively. Had consumptive use not been 

considered, bias in mean streamflow would have increased 

from 9.3 to 18.0 percent for HSPF and from 5.1 to 

22.5 percent for SWAT. 

Differences in seasonal runoff bias among models can 

be partially explained by differences in ET estimates. 

Assuming no consumptive use or deep seepage losses, 

ET can be estimated as the difference between longterm mean annual PPT and runoff (fig. 3.8A). Using the 
26

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Table 3.8—Summary of classical model fit statistics for the seven select models compared in this 

study at the five common sites from 1980–1999 

Monthly Dailyb

Site Model

Calibration 

levela

Bias in mean 

streamflow

(percent) NSE

RMSE

(cfs) R2 NSE

RMSE

(cfs) R2

1 HSPF C -6.4 0.85 145.5 0.87 0.66 353.2 0.67

1 PRMS-SERAP D -1.8 0.92 115.8 0.92 0.80 265.5 0.81

1 PRMS-DAYMET D -2.5 0.92 103.9 0.93 0.87 208.2 0.88

1 SWAT C -3.6 0.56 255.5 0.79 0.36 544.5 0.56

1 WaterFALL® C 4.6 0.90 119.0 0.91 0.59 429.1 0.67

1 MWBM D -1.4 0.89 135.3 0.89 — — —

1 WaSSI A -2.8 0.68 227.4 0.79 — — —

2 HSPF B 2.6 0.71 147.2 0.73 -0.23 543.6 0.20

2 PRMS-SERAP D 1.3 0.92 81.2 0.92 0.84 207.8 0.84

2 PRMS-DAYMET D 0.1 0.86 102.5 0.87 0.60 292.6 0.60

2 SWAT B -26.2 0.64 137.7 0.75 0.41 253.3 0.44

2 WaterFALL® C -33.7 0.62 69.5 0.87 0.31 302.5 0.37

2 MWBM D -2.1 0.84 114.2 0.84 — — —

2 WaSSI A 25.0 0.72 123.3 0.84 — — —

3 HSPF B -24.6 0.62 211.8 0.67 -0.18 855.1 0.26

3 PRMS-SERAP D -12.0 0.76 135.0 0.82 0.75 296.9 0.77

3 PRMS-DAYMET D -2.9 0.82 158.6 0.83 0.64 400.4 0.64

3 SWAT B -18.2 0.65 173.8 0.70 0.50 300.3 0.55

3 WaterFALL® B -48.0 0.28 114.2 0.61 0.38 378.1 0.43

3 MWBM D -6.6 0.85 157.7 0.86 — — —

3 WaSSI A -1.7 0.75 179.8 0.75 — — —

4 HSPF B 9.2 0.92 506.5 0.93 0.74 1408.9 0.75

4 PRMS-SERAP D 16.8 0.87 615.1 0.91 0.74 1569.1 0.75

4 PRMS-DAYMET D -0.3 0.89 577.6 0.89 0.67 1528.3 0.68

4 SWAT B 5.0 0.86 725.6 0.88 0.63 1499.7 0.64

4 WaterFALL® C 11.2 0.86 691.4 0.90 0.79 1383.5 0.80

4 MWBM D -8.4 0.83 755.5 0.85 — — —

4 WaSSI A 16.2 0.77 858.6 0.82 — — —

5 HSPF C -2.1 0.75 283.2 0.77 0.37 667.3 0.46

5 PRMS-SERAP D 3.9 0.83 235.1 0.85 0.80 370.6 0.80

5 PRMS-DAYMET D -15.3 0.73 280.9 0.82 0.47 525.1 0.50

5 SWAT C 5.8 0.46 401.9 0.77 0.24 794.5 0.52

5 WaterFALL® B -15.1 0.83 182.0 0.88 0.17 761.8 0.37

5 MWBM D -9.4 0.77 265.3 0.79 — — —

5 WaSSI A 15.4 0.69 300.9 0.81 — — —

a

 Calibration levels include: (A) uncalibrated, (B) calibrated to downstream gauge, (C) calibrated specifically for site, and (D) 

calibrated specifically for site with adjusted precipitation, solar radiation, and potential evapotranspiration (PET) inputs.

b

 The monthly MWBM and WaSSI models were not evaluated for prediction of ERFMs because many of the ERFMs require a daily 

time step to be calculated.

cfs = cubic feet per second; DAYMET = Daily Meteorological Data; IDW = Inverse Distance Weighted; MWBM = Monthly Water 

Balance Model; NR = Next-Generation Radar; NSE = Nash-Sutcliffe Efficiency; PRMS = Precipitation-Runoff Modeling System; R2 = 

coefficient of determination; RMSE = root mean squared error; SERAP = Southeast Regional Assessment Project; SWAT = Soil and 

Water Assessment Tool; WaSSI = Water Supply Stress Index model; WATER = Water Availability Tool for Environmental Resources; 

WaterFALL® = Watershed Flow and Allocation modeling system using NHDPlus. 
-60

-40

-20

0

20

40

60

1 2 3 4 5

Bias in mean flow (%)

Site

(A)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

1234 5

Monthly NSE

Site

HSPF PRMS-SERAP PRMS-DAYMET SWAT

WaterFALL® MWBM WaSSI Median for site

(B)

USDA Forest Service, Southern Research Station 27 e-General Technical Report SRS-246

CHAPTER 3

Figure 3.5—Distribution of selected classical model fit statistics across models by site for the 

flow time series from 1980–1999; bias in mean flow (A) and monthly NSE (B). 
0

1

2

3

4

5

HSP

F

PRMS-SERAP

PRMS-DAYMET

SWAT

WATERFALL®

MWBM

WASSI

Number of sites

(A)

0

1

2

3

4

5

HSP

F

PRMS-SERAP

PRMS-DAYMET

SWAT

WATERFALL

®

MWBM

WASSI

Number of sites

Very good Good Satisfactory Poor

(B)

28

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Figure 3.6—Distribution of sites falling into performance categories for absolute bias in 

mean flow (A) and monthly NSE (B) established by Moriasi and others (2007). Bias in mean 

flow within ±25, ±15, and ±10 percent is considered to be indicative of satisfactory, good, 

and very good model performance, respectively, while monthly NSE values that are >0.50, 

>0.65, and >0.75 for prediction of monthly streamflow are indicative of satisfactory, good, 

and very good model performance, respectively. 
0

10

20

30

40

50

60

Absolute bias in mean 

streamflow (%)

A

WaSSI

B

HSPF sites 2, 3, 4

SWAT sites 2, 3, 4

WaterFALL® sites 3, 5

C

HSPF sites 1, 5

SWAT sites 1, 5

WaterFALL® sites 1, 2, 4

D

PRMS-SERAP

PRMS-DAYMET

MWBM

(A)

Calibration level

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Monthly NSE

Calibration level

Site 1 Site 2 Site 3 Site 4 Site 5 Median across sites at calibration level

A

WaSSI

B

HSPF sites 2, 3, 4

SWAT sites 2, 3, 4

WaterFALL® sites 3, 5

C

HSPF sites 1, 5

SWAT sites 1, 5

WaterFALL®, sites 1, 2, 4

D

PRMS-SERAP

PRMS-DAYMET

MWBM

(B)

USDA Forest Service, Southern Research Station 29 e-General Technical Report SRS-246

CHAPTER 3

Figure 3.7—Distribution of selected classical model fit statistics across sites by level of calibration for the 1980–1999 

flow time series; absolute bias in mean streamflow (A) and monthly NSE (B). Calibration levels include A (uncalibrated), 

B (calibrated to downstream gauge), C (calibrated specifically for site), and D (calibrated specifically for site with 

adjusted precipitation, solar radiation, and PET inputs). 
0

200

400

600

800

1000

1200

1400

Precipitation

(mm)

Residual

Runoff

Evapotranspiration

Mean precipitation across

all models

(A)

Observed

HSPF

MWBM

PRMS-SERAP

PRMS-DAYMET

SWAT

WaSSI

WaterFALL®

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10 11 12

Soil moisture (%)

Month

0

10

20

30

40

50

60

70

80

1 2 3 4 5 6 7 8 9 10 11 12

Streamflow (mm)

Month

20

40

60

80

100

120

140

160

1 2 345 6 7 8 9 10 11 12

Evapotranspiration 

(mm)

Month

20

40

60

80

100

120

140

160

1 2 3 4 5 6 7 8 9 10 11 12

Precipitation (mm)

Month

HSPF MWBM PRMS-SERAP PRMS-DAYMET

SWAT WaSSI WaterFALL® Observed

(B)

30

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Figure 3.8—Partitioning of mean annual precipitation into evapotranspiration and runoff and residual lost to either 

consumptive use or deep seepage (A) and monthly median precipitation, evapotranspiration, soil moisture, and runoff (B) 

for all models at site 4: USGS gauge 02347500 Flint River at US 19 near Carsonville, GA. Runoff was computed by dividing 

discharge by drainage area. Observed ET was computed by taking the difference between the mean precipitation across 

models and the observed mean annual runoff at the gauge. 
USDA Forest Service, Southern Research Station 31 e-General Technical Report SRS-246

CHAPTER 3

mean PPT across models as the mean PPT for the basin, 

estimated ET, calculated as the difference between mean 

PPT and runoff, was 797 mm. Bias in predicted ET was 

then 0.8, -14.4, -15.6, -1.4, -16.5, -5.9, and -1.7 percent 

for HSPF, PRMS-SERAP, PRMS-DAYMET, SWAT, 

WaterFALL®, MWBM, and WaSSI, respectively. There 

was good agreement between HSPF, SWAT, WaSSI, and 

the estimated observed mean annual ET. WaterFALL®

predicted less ET relative to other models and the 

estimated observed ET, but the deep seepage term partially 

compensated for this difference. Soil moisture levels were 

considerably lower for WaterFALL® than the other models 

(fig. 3.8B), indicating that available soil water storage 

and/or recession coefficients may explain the lower ET 

estimates. The PRMS and MWBM models also underpredicted ET relative to other models and the estimated 

observed ET largely because PPT was reduced during 

model calibration.

The full suite of 175 flow metrics computed with the 

EflowStats package in R was evaluated for redundancy 

and reduced to a subset of 14 ERFMs which accounted 

for all five components of the flow regime (table 3.9). We 

evaluated differences between predicted and observed 

ERFMs across the five hourly and daily models; the 

monthly MWBM and WaSSI models were not evaluated 

for prediction of ERFMs because many of the ERFMs 

require a daily time step to be calculated (table 3.8). 

Overall bias in the prediction of the ERFMs among 

sub-monthly time-step models varied by site and by flow 

metric (fig. 3.9), with no model or calibration level clearly 

having superior predictive performance for all sites and 

metrics (table 3.10). The median absolute bias across all 

ERFMs and sites was 19.4, 19.1, 18.7, 31.9, and 24.1 

percent for HSPF, PRMS-SERAP, PRMS-DAYMET, 

SWAT, and WaterFALL®, respectively. Increasing 

calibration tended to reduce bias for individual models 

Table 3.9—Definitions of the reduced set of 14 ecologically relevant flow metrics (ERFMs) used in this study to 

describe the five primary components of the flow regime

Streamflow 

component ERFM Description Unit of measurement

Magnitude (M) MA41 Mean annual runoff: Compute the annual mean daily streamflow and 

divide by the drainage area. 

cubic feet per second (cfs) 

per square mile (cfsm)

MA25 Variability of February flow values: Compute the standard deviation for 

each month in each year. Divide the standard deviation by the mean for 

each month and take the mean of these values for each month across 

years. 

percent

ML6 Minimum June streamflow: minimum June streamflow across the 

period of record 

cfs

ML9 Minimum September streamflow: minimum September streamflow 

across the period of record 

cfs

ML21 Variability of annual minimum flows: Compute the standard deviation of 

annual minimum streamflow and divide by the mean annual minimum 

streamflow. 

percent

MH20 Specific mean annual maximum flow: Divide mean annual maximum 

flow across the period of record by watershed area. 

cfsm

Frequency (F) FL1 Frequency of low flood: low flood pulse count n/year

FL2 Variability in low-pulse count: coefficient of variation for the number of 

annual occurrences of daily flows less than the 25th percentile 

dimensionless

Duration (D) DL17 Variability in low pulse duration: standard deviation for the yearly 

average low-flow pulse durations (daily flow less than the 25th 

percentile) 

percent

DH20 High flow duration days

DL4 Mean of the annual minimum 30-day average flows cfs

DH4 Mean of the annual maximum 30-day moving average flow for the 

entire record 

cfs

Timing (T) TH1 Average Julian date of the annual maximum flow for the entire record Julian day

Rate of change (RA) RA4 Variability of the fall rate for the entire record percent

A = average; L = low or minimum flow; H = high or maximum flow.
DL4

DH4

TH1

RA4

MH20

FL1

FL2

DL17

DH20

MA41

MA25

ML6

ML9

ML21

Magnitude Frequency Duration Timing Rate of 

change

Site 1 Site 2 Site 3 Site 4 Site 5

Bias (%)

HSPF PRMS-SERAP PRMS-DAYMET SWAT WaterFALL®

-100 -50 0 50 100 150 200

32

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Figure 3.9—Bias in prediction of the 14 ERFMs across the five study sites for the daily time-step hydrologic models. [See 

table 3.9 for definitions of ERFMs.] 
USDA Forest Service, Southern Research Station 33 e-General Technical Report SRS-246

CHAPTER 3

Table 3.10—Observed and predicted ecologically relevant flow metricsa for the five sites used in this study 

Magnitude Frequency Duration Timing

Rate of 

change

Site

MA41

(cfsm)

MA25

(%)

ML6

(cfs)

ML9

(cfs)

ML21

(%)

MH20

(cfsm)

FL1

(n/

year) FL2

DL17

(%)

DH20

(days)

DL4

(cfs)

DH4

(cfs)

TH1

(Julian 

day) RA4

Observed 1 2.40 62.0 446.5 276.8 37.8 21.99 13.7 67.0 42.9 7.9 303.2 1,530.5 21 342.5

HSPF 1 2.25 53.4 411.1 320.6 31.0 15.37 9.2 56.1 60.8 9.6 293.6 1,523.7 31 249.4

PRMS-SERAP 1 2.36 50.3 444.1 289.9 50.7 14.90 9.2 64.0 43.2 8.0 271.8 1,434.5 362 240.9

PRMS-DAYMET 1 2.34 52.1 465.9 309.9 42.7 18.36 9.8 61.6 45.1 5.8 297.8 1,373.5 1 320.4

SWAT 1 2.31 57.3 127.7 119.6 92.5 16.03 11.2 55.9 36.1 4.5 113.1 1,886.1 26 185.5

WaterFALL® 1 2.51 64.7 481.8 345.7 28.7 20.68 8.4 59.8 44.9 4.2 330.1 1,524.2 13 294.9

Observed 2 1.49 90.1 92.2 51.2 62.4 17.62 9.5 40.8 45.8 5.4 59.9 1,007.3 58 269.3

HSPF 2 1.53 82.6 110.5 74.6 31.6 23.90 13.9 38.9 42.3 3.7 84.1 993.2 50 363.8

PRMS-SERAP 2 1.51 91.5 99.6 42.2 41.8 16.56 13.2 38.5 27.8 4.9 67.6 994.1 35 217.6

PRMS-DAYMET 2 1.49 75.2 99.9 64.0 35.7 17.06 13.7 48.0 32.0 4.3 66.1 959.6 23 270.6

SWAT 2 1.10 49.1 86.4 56.0 92.7 7.03 5.9 41.9 38.3 14.5 32.8 797.4 48 217.0

WaterFALL® 2 0.99 83.8 53.8 41.7 34.5 14.82 13.4 37.4 35.1 5.8 42.3 663.9 18 430.3

Observed 3 1.26 92.1 42.6 26.5 41.3 27.95 7.1 46.9 41.5 8.4 25.5 1,495.6 60 418.3

HSPF 3 0.95 105.7 52.2 39.0 36.8 31.86 8.0 49.7 43.0 6.2 37.8 1,183.2 67 740.3

PRMS-SERAP 3 1.11 89.4 72.4 56.5 34.7 18.99 11.9 41.7 27.1 6.4 70.4 1,112.5 59 338.0

PRMS-DAYMET 3 1.23 82.5 82.6 33.2 64.7 21.34 13.0 39.5 34.9 7.3 50.7 1,259.5 38 386.9

SWAT 3 1.04 61.8 156.3 77.8 97.9 9.65 3.7 48.8 61.4 14.4 44.4 1,002.9 53 273.1

WaterFALL® 3 0.66 95.5 46.3 37.3 44.8 17.71 15.1 37.3 25.5 5.6 44.9 713.3 38 519.1

Observed 4 1.12 73.0 475.3 302.5 42.9 14.79 6.9 30.5 44.4 9.4 342.8 6,650.1 52 296.9

HSPF 4 1.22 56.5 723.8 531.7 30.6 11.98 9.8 41.9 33.2 8.0 532.5 6,562.3 58 218.9

PRMS-SERAP 4 1.31 58.7 827.6 479.6 45.0 12.02 8.8 46.7 29.0 14.1 511.5 6,941.4 55 265.6

PRMS-DAYMET 4 1.12 65.1 750.2 511.5 29.8 12.88 12.2 54.2 32.5 8.3 494.9 5,948.9 44 298.4

SWAT 4 1.17 39.7 884.3 577.5 57.4 7.52 5.3 42.1 58.1 15.1 366.9 6,511.7 63 177.9

WaterFALL® 4 1.24 52.8 486.2 272.2 60.5 11.99 6.7 42.0 45.0 10.6 220.7 7,191.2 55 310.7

Observed 5 1.19 48.1 197.8 196.3 37.3 11.36 7.9 38.2 58.4 11.4 202.9 2,014.0 47 338.0

HSPF 5 1.17 58.1 310.2 249.7 40.6 10.67 6.7 59.2 84.7 13.1 198.7 1,973.4 38 278.1

PRMS-SERAP 5 1.24 42.7 287.2 234.9 24.1 8.46 7.7 47.8 34.1 10.9 232.5 1,999.4 36 268.8

PRMS-DAYMET 5 1.01 35.9 76.2 67.3 84.1 5.99 11.8 30.4 28.9 20.5 71.5 2,046.9 25 166.7

SWAT 5 1.26 62.8 200.8 180.8 66.2 11.00 4.9 48.6 66.9 16.5 109.1 2,632.4 43 243.8

WaterFALL® 5 1.01 79.3 293.2 184.8 51.4 14.04 3.8 53.0 50.7 7.6 124.4 1,719.4 29 454.4

a

 See table 3.9 for definitions of ecologically relevant flow metrics.

cfs = cubic feet per second; cfsm = cubic feet per second per square mile; DAYMET = Daily Meteorological Data; HSPF = Hydrological Simulation ProgramFortran; SERAP = Southeast Regional Assessment Project; PRMS = Precipitation-Runoff Modeling System; SERAP = Southeast Regional Assessment 

Project; SWAT = Soil and Water Assessment Tool; WaterFALL® = Watershed Flow and Allocation modeling system using NHDPlus.
overall, with median absolute bias across all sites and 

ERFMs decreasing from calibration level B to level C for 

HSPF (22.6 percent for level B, 16.9 percent for level C), 

SWAT (36.2 percent for level B, 27.1 percent for level C), 

and WaterFALL® (36.7 percent for level B, 14.9 percent 

for level C). The median absolute bias across all sites and 

ERFMs for calibration level D (19.1 percent) was similar 

to that of all models at calibration level C (18.4 percent). 

All models had at least one flow metric falling outside 

the ±30-percent range of hydrologic uncertainty at every 

site (figs. 3.9 and 3.10A). The number of ERFMs out 

of the total of 14 (table 3.9) that fell outside this range 

at three or more of the five sites included three, four, 

five, nine, and five for HSPF, PRMS-SERAP, PRMSDAYMET, SWAT, and WaterFALL®, respectively. Some 

of the magnitude, frequency, and duration ERFMs tended 

to be better represented across models than others. For 

example, MA41 (mean annual runoff), FL2 (variability 

in low-pulse count), and DH4 (mean of annual maximum 

30-day average flow) were generally well predicted, with 

bias outside the range of uncertainty at less than three of 

the five sites for all models. Bias in FL1 (frequency of 

low flood), however, was outside the range of uncertainty 

for three or more of the five sites for all models, and 

bias in ML21 (variability of annual minimum flows) and 

DL4 (mean of annual minimum 30-day average flows) 

was outside the range of uncertainty for three or more of 

the five sites for four of the five models. The hydrologic 

models evaluated in this study generally had lower bias 

in the prediction of flow metrics representing mean flows 

(e.g., MA41) than metrics representing the extremes of 

flow (e.g., FL1), particularly low-flow conditions. Bias 

was greater for many of the low-flow metrics due to the 

low absolute magnitudes of these metrics (table 3.10, 

fig. 3.9). This result is a fairly common outcome for 

many modeling studies due to choices made during the 

calibration process; however, it should be noted that this 

modeling bias may directly affect the predictive capacity 

of flow-ecology response models derived using ERFMs 

that fall outside the established range of uncertainty. 

There was considerable variability in ERFM predictive 

performance across sites (table 3.10, figs. 3.9 and 3.10B). 

The ERFMs (table 3.9) with prediction bias outside the 

range of hydrologic uncertainty for three or more of the 

five models included FL1 and TH1 for site 1 (two of 14 

ERFMs); ML21, FL1, and TH1 for site 2 (three of 14); 

ML6, ML9, MH20, FL1, DL17, and DL4 for site 3 (six 

of 14); ML6, ML9, ML21, FL2, and DL4 for site 4 (five 

of 14); and ML6, ML21, FL1, DL17, DH20, and DL4 for 

site 5 (six of 14). The ERFMs at site 1 were generally well 

predicted, likely reflecting the fact that (1) all models were 

calibrated for this site (i.e., calibration levels C and D), 

MA41 0 0 0 0 2

MA25 0 0 0 4 1 0

ML6 2 3 3 3 2 0

ML9 3 2 2 3 1 1

ML21 1 3 4 5 3 1

MH20 2 2 1 3 1 2

FL1 3 3 4 3 4 2

FL2 2 1 1 1 2 3

DL17 2 4 2 2 1 3

DH20 1 1 1 5 3 4

DL4 3 2 3 4 3 4

DH4 0 0 0 2 2 5

TH1 1 2 4 0 4 5

RA4 2 0 1 3 2 HSPF PRMS- SERAP PRMS- DAYMET SWAT WATERFALL®

5

0

1

2

3

4

(A)

MA41 0 0 0 0 2

MA25 0 0 0 4 1 0

ML6 2 3 3 3 2 0

ML9 3 2 2 3 1 1

ML21 1 3 4 5 3 1

MH20 2 2 1 3 1 2

FL1 3 3 4 3 4 2

FL2 2 1 1 1 2 3

DL17 2 4 2 2 1 3

DH20 1 1 1 5 3 4

DL4 3 2 3 4 3 4

DH4 0 0 0 2 2 5

TH1 1 2 4 0 4 5

RA4 2 0 1 3 2 HSPF PRMS- SERAP PRMS- DAYMET SWAT WATERFALL®

5

0

1

2

3

4

(A)

MA41 0 1 1 0 0

MA25 0 1 1 1 2

ML6 1 1 3 4 4

ML9 1 1 4 4 1

ML21 2 5 2 3 4

MH20 2 2 3 1 1

FL1 3 5 4 2 3

FL2 0 0 0 5 2

DL17 1 2 3 2 3

DH20 2 2 2 2 3

DL4 1 2 5 4 3

DH4 0 1 2 0 1

TH1 4 3 2 0 2

RA4 1 2 2 1 2 Site 1 Site 2 Site 3 Site 4 Site 5

(B)

MA41 0 1 1 0 0

MA25 0 1 1 1 2

ML6 1 1 3 4 4

ML9 1 1 4 4 1

ML21 2 5 2 3 4

MH20 2 2 3 1 1

FL1 3 5 4 2 3

FL2 0 0 0 5 2

DL17 1 2 3 2 3

DH20 2 2 2 2 3

DL4 1 2 5 4 3

DH4 0 1 2 0 1

TH1 4 3 2 0 2

RA4 1 2 2 1 2 Site 1 Site 2 Site 3 Site 4 Site 5

(B)

MA41 0 0 0 0 2

MA25 0 0 0 4 1 0

ML6 2 3 3 3 2 0

ML9 3 2 2 3 1 1

ML21 1 3 4 5 3 1

MH20 2 2 1 3 1 2

FL1 3 3 4 3 4 2

FL2 2 1 1 1 2 3

DL17 2 4 2 2 1 3

DH20 1 1 1 5 3 4

DL4 3 2 3 4 3 4

DH4 0 0 0 2 2 5

TH1 1 2 4 0 4 5

RA4 2 0 1 3 2 HSPF PRMS- SERAP PRMS- DAYMET SWAT WATERFALL®

5

0

1

2

3

4

(A)

34

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Figure 3.10—Number of sites per model out of a total of five (A) 

and number of models per site out of a total of five (B) for which 

bias in the 14 ERFMs fell outside of the ±30-percent range of 

hydrologic uncertainty. Monthly models (MWBM and WaSSI) are 

not shown because computation of most ERFMs used in this 

study requires a daily time step. [See table 3.2 for hydrologic 

model definitions and table 3.9 for definitions of ERFMs.]
USDA Forest Service, Southern Research Station 35 e-General Technical Report SRS-246

CHAPTER 3

and (2) the watershed upstream of the site was mostly lowintensity forested land (table 3.4) and likely had fewer flow 

alterations that may affect model performance. Although 

sites 1 and 3 had similar levels of urban development, 

ERFMs were not as well predicted at site 3. In particular, 

those ERFMs relating to low flows (e.g., ML6, ML9, FL1, 

DL17, and DL4) were generally over-predicted with bias 

exceeding 30 percent (fig. 3.10B). The HSPF, SWAT, and 

WaterFALL® models were not specifically calibrated for 

site 3 (calibration level B), which may explain the higher 

bias for this site, but the PRMS-SERAP and PRMSDAYMET models, calibrated specifically for this site, 

(calibration level D) also had higher bias for ERFMs at 

site 3 than site 1, indicating that there may be underlying 

natural (e.g., higher than expected ET losses) and/or 

anthropogenic processes (e.g., surface and groundwater 

use, interbasin transfers, water diversions, etc.) that are 

not being accounted for in the models despite the fact 

that these sites are assumed to be minimally altered. Most 

model predictions of ERFMs for site 2 were within the 

range of uncertainty despite having higher levels of urban 

development (table 3.4), except that ML21 (variability 

in annual minimum flows) was generally underpredicted, FL1 (frequency of low flood) was generally 

over-predicted, and TH1 (average Julian date of annual 

maximum flow) was generally under-predicted (table 3.10, 

fig. 3.9). The ERFMs relating to low flows were generally 

over-predicted by most models for site 4 (e.g., ML6, ML9, 

FL2, and DL4) and site 5 (e.g., ML6 and ML21). 

DISCUSSION

The intent of this model comparison study was not to 

suggest that the performance of any particular model is 

superior to that of the others. Rather, we were interested in 

understanding differences among hydrologic models and 

calibration strategies by quantifying and comparing the 

potential causes of error associated with model prediction 

and testing our hypotheses that (1) in general, regionalscale hydrologic models (e.g., MWBM, WaSSI) would 

have poorer predictive capacity and higher levels of 

uncertainty than the fine-scale models (e.g., HSPF, PRMS, 

SWAT, WaterFALL®); and (2) models with higher levels 

of calibration would perform better than those that were 

less calibrated. In order to accomplish these objectives, we 

summarized a subset of classical model fit statistics (e.g., 

mean bias, R2, RMSE, and NSE) for seven hydrologic 

models of varying calibration intensity across five study 

sites where modeling efforts overlapped with USGS 

continuous record gauges. 

We found that all models had “satisfactory” or better 

performance (as defined in this paper) at most sites. 

Comparing classical model fit statistics across all sites, 

the broad-scale MWBM and WaSSI had comparable 

error in predicting observed streamflows at the monthly 

time step as that of the fine-scale HSPF, PRMS, SWAT, 

and WaterFALL® models (fig. 3.5), which refutes our 

hypothesis that regional-scale models have poorer 

predictive performance than fine-scaled models at the 

monthly time step. For example, according to monthly 

NSE criteria (Moriasi and others 2007), the uncalibrated 

WaSSI model predictions would be considered “good” at 

all of the five sites simulated by multiple models and “very 

good” at one site, while the calibrated MWBM predictions 

would be “very good” at all sites (fig. 3.6B). Achieving 

good model fit at the monthly time step with either 

monthly or smaller time-step models generally indicates 

that the correct balance of PPT and ET is represented, but 

it does not necessarily indicate that the separation between 

surface and subsurface flows is accurately represented. 

Thus, good model fit in monthly time-step simulations 

may not indicate that the model would be useful in 

answering resource questions that require detailed 

information regarding surface runoff and baseflows.

We also found evidence supporting our hypothesis that 

increasing calibration intensity generally improved 

model fit across sites and models. For classical model 

fit statistics, the more intensive site-specific calibrations 

(levels C and D) generally decreased bias and increased 

NSE at the monthly scale relative to uncalibrated models 

(level A) and models calibrated to a downstream site 

(level B); however, differences between calibration 

levels A and B were not as large (fig. 3.7). For ERFMs, 

increasing calibration tended to reduce bias for individual 

models overall, but no model or calibration level clearly 

had superior predictive performance for all sites and 

ERFMs. For example, bias in ERFM predictions was 

generally lower for sites calibrated at level C than for sites 

calibrated at level B for HSPF, SWAT, and WaterFALL®. 

However, differences in ERFM bias between models 

calibrated for sites at level C and models calibrated for 

sites at level D were generally smaller. Clearly, adjusting 

PPT, solar radiation, and PET during model calibration 

(i.e., calibration level D) can result in improved model fit 

relative to other levels of calibration; however, caution 

should be used when applying models calibrated in 

this manner to make projections using other sources of 

climate input (e.g., future climate change scenarios). For 

example, it is common to use a climate dataset based on 

historical weather observations to calibrate a model and 

to use one or more downscaled climate datasets produced 

by General Circulation Models (GCMs) for future 

scenarios. Often the GCM-predicted climate variables will 

differ from observations in magnitude and distribution 

over the historical period. If historical observed climate 

data are adjusted in the calibration process to achieve 

a better model fit to streamflow observations, then a 

modeler would need to determine how to incorporate 
36

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

this adjustment into the model when the GCM-predicted 

climate variable over the historical period may be 

different than the observed variable used to derive the 

adjustment factors. While this could be done, the modeler 

would need to exercise caution in doing so. In addition 

to model calibration, model inputs and assumptions also 

played a role in predictive performance. For example, the 

WaterFALL® model used land cover from the 1970s (with 

lower levels of impervious cover than in 2001 or 2006 

used by other models) and tended to have negative bias for 

sites with some level of urbanization (e.g., site 2) because 

surface runoff was lower. 

Similar to other ecological flow modeling studies 

(Murphy and others 2013, Wenger and others 2010), 

the sub-monthly time-step models evaluated in this 

study tended to have “good” predictive performance for 

ERFMs representing the mean of flow, but had difficulty 

in predicting flow metrics related to low or extreme 

flows (table 3.10, figs. 3.9 and 3.10). The variability in 

prediction bias across ERFMs for the sub-monthly models 

is indicative of the variability in level of calibration 

across models and sites but also the challenges associated 

with calibrating hydrologic models for all streamflow 

conditions. Model calibration is generally intended to 

capture the variability and mean magnitude of streamflow. 

It is nearly impossible to calibrate models to fit the entire 

range of observed streamflows because adjusting model 

parameters to fit a portion of the flow regime has an 

effect on how well the model fits observed streamflows 

outside of that range. For example, WaterFALL® was 

calibrated to log-transformed daily streamflows (table 3.5) 

to provide improved fit for low flows, but calibrating in 

this way can degrade fit for high or median flows. There 

was considerable variability in classical fit statistics and 

ERFM predictive performance across sites (figs. 3.5, 3.9, 

and 3.10). Fit statistics and ERFMs at some sites (e.g., 

site 1) were better predicted by all models than other sites 

(e.g., site 3), illustrating the fact that model performance 

is site-specific regardless of model framework or level of 

calibration. These findings may have implications for the 

development of flow-ecology response models because 

it is often the low flows (baseflows), annual-flow pulses, 

and seasonality of high flows that provide the conditions 

necessary to support natural-assemblage complexity 

(Matthews 2005, Poff and Ward 1989, Poff and others 

1997, Richter and others 1997, Stanford and others 1996). 

Because streamflow models in this study tended to perform 

better when predicting mean ERFMs than when predicting 

low or extreme flow ERFMs, great care should be taken 

when using ERFMs with high prediction bias (e.g., ML6, 

ML9, FL1, DL17, and DL4) to develop flow-ecology 

response models. 

Other approaches to predicting ERFMs (e.g., regression 

models) have been recently shown to have better 

predictive performance for low-flow metrics (Knight and 

others 2012, Murphy and others 2013). However, rainfallrunoff and other physically based hydrologic models 

are still needed for evaluating environmental change 

and hydrologic alteration effects on aquatic ecosystems 

because they are more flexible and can simulate scenarios 

of change. Additionally, even though some regression 

models appear to perform well in parts of the SEUS, it is 

difficult to predict whether they will show the same level 

of performance or have a high level of transferability in 

the snowmelt-driven Rocky Mountain States or in areas 

such as the Southwestern United States where low flows 

predominate and where there are fewer gauges available to 

establish statistical relations. While rainfall-runoff models 

are more flexible and can simulate scenarios of change, 

it should be kept in mind that all hydrologic models, 

regardless of their level of complexity, are simplified 

mathematical representations of natural systems and 

therefore may not adequately reflect all of the processes 

that affect streamflow and/or may not adequately predict 

streamflow response to environmental change. Further, 

uncertainty of hydrologic model predictions are inherently 

dependent on the uncertainty of model inputs (e.g., soils, 

land cover, climate).

LIMITATIONS

While this report likely represents what may be the most 

rigorous evaluation of the use of hydrologic models 

for flow-ecology science compiled for the SEUS, we 

acknowledge that it may provide a limited overview of the 

many tools and techniques available. Some limitations of 

this study include:

1. Only a subset of the available hydrologic models were 

considered in this study.

2. The models evaluated were not developed using 

the same calibration objective functions and input 

datasets, making it difficult to separate differences 

in model performance that may be related to the 

model framework (e.g., HSPF or PRMS) from 

differences associated with the choice of model inputs 

and calibration.

3. New methods are being developed all the time. The 

findings in this report are not stationary and should be 

reevaluated from time to time.

4. The capacity to fully understand non-stationary 

conditions associated with climate change requires 

rigorous calibration of models and careful attention to 

model inputs and representation of physical processes 

that may assume stationarity. For example, adjusting 

PPT and other climate variables may improve model fit 
USDA Forest Service, Southern Research Station 37 e-General Technical Report SRS-246

CHAPTER 3

for historical flow observations, but these adjustments 

may not be appropriate when using the model to make 

projections using other sources of climate input (e.g., 

future climate change scenarios).

5. Models evaluated were applied to several basins 

in SEUS region. The relative model performance 

may be different in other hydroclimatic settings 

(e.g., snowmelt-dominated streams and streams in 

arid climates). 

6. There are tradeoffs between regional- and fine-scale 

models. Regional-scale models are often easier to 

parameterize but are often of a coarser resolution 

in space and time. Fine-scale models are often 

more difficult to parameterize but can have finer 

resolution and smaller time steps. Resource managers 

should consider the desired resolution of streamflow 

predictions when selecting a model for addressing 

a particular resource problem and balance the need 

for that resolution with the expense in input data 

requirements, computational limitations, and desired 

level of uncertainty. 

7. In this study, we have demonstrated that similar levels 

of model performance may be obtained at the monthly 

time step using regional- and fine-scale hydrologic 

models. We have also shown that regional-scale and 

some fine-scale hydrologic models predict similar 

changes in runoff for a given change in climate inputs. 

However, these results only provide a demonstration 

of the potential of multi-scale modeling approaches to 

evaluate environmental change effects on streamflow 

and ecological response. Due to differences in land 

cover input data, we were not able to determine 

whether there would be similar streamflow response 

to land cover change among the models evaluated. 

Additional study is required to determine the best way 

to use regional- and fine-scale models to identify hot 

spots and develop flow-ecology relations, respectively. 

Models should be developed using the same inputs 

and calibration objective functions and then used 

to evaluate the same scenarios of climate and land 

cover change. 

8. There are tradeoffs between calibrating models to best 

match observed high-flow or low-flow portions of the 

hydrograph that may affect flow-ecology modeling. 

Model calibration is generally intended to capture the 

variability and the central tendency of streamflow. It is 

nearly impossible to calibrate models to fit the entire 

range of observed streamflows because adjusting model 

parameters to fit a portion of the flow regime has an 

effect on how well the model fits observed streamflows 

outside of that range. For example, it is fairly common 

for many modeling studies to have relatively large 

biases in the prediction of low-flow ERFMs due to 

choices made during the calibration process; however, 

it should be noted that this modeling bias may directly 

affect the predictive capacity of flow-ecology response 

models derived using ERFMs that fall outside the 

established range of uncertainty.

Despite these limitations, it is our hope that this rigorous 

approach to understanding differences in streamflow 

predictions among a subset of hydrologic models currently 

in use in the SEUS and developing flow-ecology response 

models will provide water resource managers and 

stakeholders with an informed pathway for developing the 

capacity to link streamflow and ecological response and an 

understanding of some of the limitations associated with 

these type of modeling efforts. 

CONCLUSIONS

The primary objective of this study was to provide 

resource managers and environmental flow practitioners 

with some insight into the relative error in streamflow 

predictions among a subset of hydrologic models 

commonly used for water supply assessment, 

environmental flow studies, and climate change 

predictions. All of the models evaluated were developed 

by different agencies, for different purposes, with 

different input datasets, and, in general, were calibrated 

to different degrees using different objective functions. 

As a result, we could not separate the relative effect of 

model structure on prediction error from that of model 

calibration and modeler expertise. To fully evaluate the 

effect of model structure alone, all models should be 

developed using the same inputs and calibrated to meet 

the same objective functions. However, our results do not 

indicate that any specific hydrologic model is superior 

to the others evaluated at all sites and for all measures of 

model performance, and do not support the hypothesis that 

regional-scale models have less predictive power than finescale models at a monthly time step. Differences among 

model predictions for specific fit statistics or ERFMs are 

as likely to be related to differences in model calibration 

strategy as they are related to differences in model 

structure. As a result, we do not provide recommendations 

of one hydrologic model over another based on the results 

of this study. Instead we stress that it is incumbent upon 

resource managers, environmental flow practitioners, and 

policymakers to consider the expertise of the modeler, the 

applicability of a model to a particular resource problem, 

the context to which the model is being applied, the scale 

of interest, and the important components of the flow 

regime that may be used for model calibration to minimize 

error across the targeted range of flows and thus improve 

flow-ecology relations.
38

CHAPTER 4

Feasibility of Combining Regional- and Local-Scale Models to 

Identify Unique Areas of Concern and Understand Fine-Scale 

Hydrologic Dynamics Under Climate Change

INTRODUCTION

Leveraging the benefits of both large-scale models and 

high-resolution models has the potential to allow more 

robust environmental change assessment studies to balance 

water resources needed to support aquatic assemblages 

while conserving water for long-term human needs across 

broad regions. For example, the Water Supply Stress 

Index (WaSSI) model, a regional-scale monthly water 

balance and flow routing model (Caldwell and others 

2012, Sun and others 2011b) is used to evaluate the 

effects of environmental change on water supply and river 

flows. The model is run uncalibrated using off-the-shelf 

databases and thus could be used to more broadly assess 

environmental change effects and identify specific areas 

of concern (“hot spots”) where the combined effects of 

land cover change, climate change, and/or flow alteration 

may threaten water resources. Fine-scale, physically 

based models of higher temporal resolution, such as the 

Hydrological Simulation Program-Fortran (HSPF) and 

the Soil and Water Assessment Tool (SWAT), could then 

be applied to those areas of concern to provide higher 

resolution quantitative estimates of changes in water 

supply and ecologically relevant flow metrics (ERFMs) 

using more site-specific inputs. To apply such a multiscale modeling approach, the variability of predicted 

streamflow response to different stressors across large- and 

fine-scale models must be assessed. For example, for a 

given change in precipitation (PPT), do the models predict 

similar changes in streamflow? The aim of this study 

was to examine the potential for combined application 

of large- and fine-scale hydrologic modeling approaches 

over large regions for climate change assessment studies. 

This was done by quantifying differences in sensitivity to 

climate change (i.e., PPT and temperature [TEMP]) among 

hydrologic models. 

METHODS

Study Site

The study site chosen for this assessment was U.S. 

Geological Survey (USGS) gauge 02347500 (Flint River at 

US 19, near Carsonville, GA) located in the ApalachicolaChattahoochee-Flint (ACF) Basin (fig. 4.1). The upstream 

drainage area of this site is 4792 km2

, with mean annual 

PPT and TEMP of 1282 mm and 16.8 °C, respectively. 

Predominant land cover consists of 13.9 percent developed 

land (3.6 percent impervious), 55 percent forest, and 16 

percent agriculture, with woody wetland (5.6 percent), 

grassland (5.3 percent), shrubland (2.6 percent), open 

water (1.4 percent), and barren (0.2 percent) land covers 

comprising the remaining area of the basin (Falcone and 

others 2010, 2011). The headwaters of this basin drain 

portions of the city of Atlanta, GA. The ACF Basin has 

been subject to water shortages and development pressure 

in the past and thus has been intensely studied over the 

last decade with multiple modeling efforts taking place 

to evaluate drought and environmental change effects on 

water supply and aquatic ecosystems (e.g., Freeman and 

others 2013, Georgakakos and others 2010, LaFontaine 

and others 2013).

Model Descriptions

Models included in this study were among those discussed 

and compared in the model comparison workshop (see 

chap. 3): Hydrological Simulation Program-Fortran 

(HSPF), the Soil and Water Assessment Tool (SWAT), the 

Generalized Watershed Loading Function (GWLF)-based 

WaterFALL® model developed by Research Triangle 

Institute (RTI), and the U.S. Department of Agriculture 

(USDA) Forest Service Water Supply Stress Index 

(WaSSI) model (table 4.1). The two parameterizations of 

Precipitation-Runoff Modeling System (PRMS) and the 

Monthly Water Balance Model (MWBM) (see chap. 3) 

were not included in this analysis because PPT and TEMP 

inputs for these models were adjusted in the calibration 

process and thus were not comparable to the climate 

inputs of HSPF, SWAT, WaterFALL®, and WaSSI. The 

WaSSI model is a regional, large-scale model, while 

HSPF, SWAT, and WaterFALL® are more complex, 

highly parameterized fine-scale models. All of the models 

evaluated in this study were developed by different 

agencies, for different purposes, and calibrated to different 

degrees using different objective functions as described 

below (Caldwell and others 2015). 

HSPF and SWAT

Both the HSPF and SWAT models used in this study 

were implemented by Tetra Tech as part of a larger study 

to characterize the sensitivity of streamflow, nutrient 

loading, and sediment loading to a range of potential 
!

!

! !

!

! !

!

!

!

!

!

! !

#

Tampa

Athens

Atlanta Augusta

Columbus Savannah

Clearwater

Montgomery

Birmingham

Huntsville

Gainesville

Tallahassee

Jacksonville

GA

FL

AL

SC

TN NC

 Legend

# Gauge station

Gauged basin

Apalachicola-Chattahoochee-Flint Basin

 ̄ 0 10050 km

USDA Forest Service, Southern Research Station 39 e-General Technical Report SRS-246

CHAPTER 4

Figure 4.1—Map of the Apalachicola-Chattahoochee-Flint Basin highlighting 

the location of the Flint River at US 19 near Carsonville, GA (USGS gauge 

02347500).

Table 4.1—General summary of hydrologic model attributes 

Model HSPF SWAT WaterFALL® WaSSI

Time step hourly daily daily monthly

Spatial resolution HUC-10

(~410 km2

)

HUC-10

(~410 km2)

NHDPlus catchment 

(~1.0 km2)

HUC-12

(~80 km2)

Withdrawals, flow regulation 

 simulated

yes yes no no

Land cover input 2001 NLCD 

(Homer and others 

2007)

2001 NLCD (Homer 

and others 2007)

ca. 1970s USGS 

GIRAS (Price and 

others 2006)

2006 NLCD (Fry and 

others 2011)

Climate Input Station 

observations

Station observations USDA (Di Luzio and 

others 2008)

PRISM (PRISM 

Climate Group 2013)

GIRAS = Geographic Information Retrieval and Analysis System; HSPF = Hydrological Simulation Program-Fortran; HUC = Hydrologic Unit Code; 

NHD = National Hydrography Dataset; NLCD = National Land Cover Database; PRISM = Parameter-elevation Relationships on Independent Slopes 

Model; SWAT = Soil and Water Assessment Tool; USDA = U.S. Department of Agriculture; USGS = U.S. Geological Survey; WaSSI = Water Supply 

Stress Index model; WaterFALL® = Watershed Flow and Allocation modeling system using NHDPlus.
40

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

mid-21st century climate futures in 20 large U.S. drainage 

basins (Johnson and others 2012, USEPA 2013). Model 

descriptions and information pertinent to this application 

are detailed below.

HSPF—The HSPF (Bicknell and others 2001, 2005) is 

a hydrology and water quality model commonly used for 

determination of Total Maximum Daily Loads to receiving 

waters in response to the Clean Water Act. HSPF is a 

well-documented watershed model that computes the 

water balance based on the Stanford Watershed Model 

(Crawford and Linsley 1966) in multiple surface and 

subsurface layers at an hourly time step. The water balance 

is simulated based on Philip’s infiltration (Bicknell and 

others 2001, 2005) coupled with multiple surface and 

subsurface stores (interception storage, surface storage, 

upper zone soil storage, lower zone soil storage, active 

groundwater, and inactive [deep] groundwater). Individual 

land units within a sub-basin are represented using a 

hydrologic response unit (HRU) approach that combines 

an overlay of land cover, soil, and slope characteristics. 

The stream network links the surface runoff and 

groundwater flow contributions from each of the HRUs 

and routes them through water bodies. The stream model 

includes PPT and evaporation from the water surfaces 

as well as streamflow contributions from the watershed, 

tributaries, and upstream stream reaches. 

SWAT—SWAT was developed to simulate the effect 

of land management practices on water, sediment, 

and agricultural chemical yields in large, complex 

watersheds with varying soils, land use, and management 

conditions over long periods of time (Neitsch and others 

2005). SWAT requires data inputs for weather, soils, 

topography, vegetation, and land use to model water and 

sediment movement, nutrient cycling, and numerous 

other watershed processes. SWAT (as implemented 

here) uses the curve number approach (USDA Soil 

Conservation Service 1972) to estimate surface runoff 

and then completes the water balance through simulation 

of subsurface flows, evapotranspiration (ET), soil 

storages, and deep seepage losses at the daily time step. 

The curve number is estimated as a function of land use, 

cover, condition, hydrologic soil group, and antecedent 

soil moisture. 

HSPF and SWAT for this application—For both models, 

the 20 larger watersheds were divided into a series of 

sub-basins at approximately the Hydrologic Unit Code 

(HUC) 10-digit scale, representing the drainage areas that 

contribute to each of the stream reaches. Both the HSPF 

and SWAT models used the 2001 National Land Cover 

Database (NLCD) (Homer and others 2007) to characterize 

the land surface. For HSPF, soils are distinguished on the 

basis of hydrologic soil group (HSG) as defined in State 

Soil Geographic (STATSGO) database (USDA NRCS 

2012) soil coverages. The HRU definitions for SWAT 

in this application use the same land cover as HSPF but 

distinguish soils based on STATSGO’s dominant soil 

classification, not just HSG. Withdrawals were included 

if they resulted in a modification of flow at downstream 

gauges on the order of 10 percent or more. Time series 

of observed PPT and air TEMP (hourly for HSPF, daily 

for SWAT) from 37 weather stations in the ACF Basin 

were obtained from the 2006 BASINS 4 Meteorological 

Database (USEPA 2008). Potential evapotranspiration 

(PET) for both HSPF and SWAT was computed using the 

Penman-Monteith energy balance method (Jensen and 

others 1990, Monteith 1965) using solar radiation, wind 

movement, cloud cover, and relative humidity estimated 

using the SWAT weather generator. A full energy balance 

approach was used because the focus of the study was 

to evaluate potential response to future climates, in 

which the relations between different energy inputs may 

change, even though a better calibration fit to current 

climate conditions can often be obtained using simpler 

temperature-based approaches when the energy inputs are 

subject to uncertainty. 

The calibration objectives for both HSPF and SWAT were 

to achieve error statistics for total streamflow volume, 

seasonal streamflow volume, and high and low streamflow 

within recommended ranges (Donigian 2000, Lumb and 

others 1994) while also maximizing the Nash-Sutcliffe 

Efficiency (NSE) (Nash and Sutcliffe 1970). Because the 

objectives of this application focused at the large basin 

scale, calibration was undertaken only at the HUC 8-digit 

and larger watershed scale. For this application of HSPF, 

four model parameters were the primary focus during 

model calibration to improve model fit for hydrology: 

INFILT (index to mean soil infiltration rate), AGWRC 

(groundwater recession rate), LZSN (lower zone nominal 

soil moisture storage), and BASETP (ET by riparian 

vegetation). For SWAT, 11 model parameters were 

adjusted during model calibration to improve model fit in 

this application: curve number; SECO (soil evaporation 

compensation factor); SURLAG (surface runoff lag 

coefficient); groundwater “revap” rates, baseflow factor; 

GW_DELAY (groundwater delay time); GWQMN 

(threshold depth of water in the shallow aquifer required 

for return flow to occur); RevapMN (threshold depth 

of water in the shallow aquifer required for “revap” or 

percolation to the deep aquifer to occur); CANMAX 

(maximum canopy storage); Manning’s “n” value for 

overland flow, main channels, and tributary channels; and 

Sol_AWC (available water capacity of the soil layer, mm 

water/mm of soil). 

WaterFALL®

Research Triangle Institute’s WaterFALL® system employs 

an updated version of a well-established hydrologic 

model, the Generalized Water Loading Function (GWLF) 
USDA Forest Service, Southern Research Station 41 e-General Technical Report SRS-246

CHAPTER 4

(Haith and Shoemaker 1987, Haith and others 1992) that 

has been modified to (1) run on the U.S. Environmental 

Protection Agency’s enhanced National Hydrography 

Dataset (NHDPlus) hydrologic network, (2) accept 

parameterization from national datasets, and (3) include 

the impacts of human alterations on streamflows. 

WaterFALL® utilizes the hydrologic simulation within 

GWLF for each catchment of the NHDPlus network in 

the watershed of interest and then accumulates and moves 

water downstream with an embedded time-lagged routing 

routine, providing a distributed hydrologic model across 

the NHDPlus. Parameterization of WaterFALL® occurs 

through the geoprocessing of national datasets for land 

cover, soils, and climate (mean daily TEMP and PPT) 

to each NHDPlus catchment. Each land cover class in a 

catchment is characterized by the predominant HSG and 

predominant percent sand, silt, and clay for the underlying 

soil forming an HRU for runoff calculation within the 

catchment. Mean TEMP and PPT are quantified by 

catchment. Additional basin-specific characteristics such 

as water uses may also be indexed to each catchment from 

local data sources. 

Like SWAT, surface runoff in WaterFALL® is computed on 

a daily basis using the curve number method across each 

land cover type in a catchment. Discharge from shallow 

groundwater is computed using a lumped parameter 

catchment-level water balance for unsaturated and shallow 

saturated zones controlled by the available water capacity 

(AWC) of the unsaturated zone, a recession coefficient 

(RCoeff) providing the rate of release from the saturated 

zone to the stream channel, and a first-order approximation 

of infiltration losses to deep aquifer storage simulated 

using a seepage coefficient (SEEP). The seepage release 

constitutes a loss from the system, where the water is 

no longer available to reach the stream in the temporal 

context of daily rainfall-runoff modeling. Daily ET from 

the unsaturated zone is computed using a land usebased cover factor and PET computed using the Hamon 

temperature-based method (Hamon 1963). Three model 

parameters (AWC, RCoeff, and SEEP) are adjusted during 

an automated calibration process using a customized 

version of the Parameter Estimation Tool (PEST). Because 

of the physical basis of the AWC and RCoeff parameters, 

a priori values for the parameters are indexed to individual 

catchments within the WaterFALL® database, and a 

multiplier across the physically based values is adjusted 

during calibration. The SEEP parameter is set through 

calibration and consideration of local conditions.

The application of WaterFALL® used for this study was 

developed to create a hydrologic foundation for detailed 

assessment of human and climate effects on stream and 

river flows, including the impacts of hydrologic alterations 

on aquatic habitats in the Southeastern United States 

(SEUS) at the NHDPlus catchment scale (~1.0 km2

) 

(Kendy and others 2011). For this study, climate inputs 

included daily PPT and TEMP from a 4- x 4-km national 

dataset obtained from the USDA (Di Luzio and others 

2008), land cover was based on the baseline condition 

assessment represented by the USGS Geographic 

Information Retrieval and Analysis System (GIRAS) land 

cover (Price and others 2006), and soils data were obtained 

from the Soil Survey Geographic database (SSURGO) 

(https://www.nrcs.usda.gov/wps/portal/nrcs/detail/soils/

survey/geo/?cid=nrcs142p2_053627). Although water 

system discharges and withdrawals were obtained from 

State databases on public and non-public systems for other 

scenarios in the SEUS study, the WaterFALL® model 

simulations contributing to this study were unaltered by 

humans. WaterFALL® was calibrated at several USGS 

gauge locations in the study area for periods in the 1970s, 

commensurate with the land use coverage used to simulate 

a less-altered baseline for the SEUS study. Although 

performance was validated with later periods, some 

differentiation from the other models is expected where 

the calibration period more closely matched the period of 

this study. The three model calibration parameters (AWC, 

RCoeff, and SEEP) were optimized to minimize the 

differences in log-transformed daily flows, giving equal 

weight to differences in streamflows at the low and high 

end of the hydrograph. 

WaSSI

The WaSSI model was developed by the Forest Service to 

assess the effects of climate change, land use change, and 

population growth on water supply stress, river flows, and 

aquatic ecosystems across the conterminous United States 

(Caldwell and others 2012, Sun and others 2011b). WaSSI 

has been successfully used in climate change assessments 

in the Eastern United States (Lockaby and others 2011, 

Marion and others 2013, Sun and others 2013, Tavernia 

and others 2013) and for examining the nexus of water 

and energy at the national scale (Averyt and others 2011, 

2013). WaSSI is an integrated monthly water balance and 

flow routing model that simulates the full hydrologic cycle 

for each of 10 land cover classes at the 12-digit Hydrologic 

Unit Code (HUC) scale. The 10 land cover classes are 

aggregated from the 17 classes of the 2006 NLCD (Fry 

and others 2011). Infiltration, surface runoff, soil moisture, 

and baseflow processes for each HUC watershed’s land 

cover were computed using algorithms of the Sacramento 

Soil Moisture Accounting Model (SAC-SMA) (Burnash 

1995, Burnash and others 1973). STATSGO databases 

(USDA NRCS 2012) were used to compute the 11 SACSMA soil input parameters (Koren and others 2003). 

Monthly ET was modeled with an empirical equation 

derived from multisite eddy covariance ET measurements 

(Sun and others 2011a, 2011b). Required data to estimate 

ET included monthly mean Moderate Resolution Imaging 
42

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Spectroradiometer (MODIS) MOD15A2 leaf area index 

(LAI) (Zhao and others 2005), Hamon PET calculated 

as a function of TEMP and latitude (Hamon 1963), and 

PPT. This estimate of ET was then constrained by the 

soil water content computed by the SAC-SMA algorithm 

during extreme water-limited conditions. Monthly PPT and 

air TEMP inputs were based on Precipitation Elevation 

Regression on Independent Slopes Model (PRISM) 

estimates (PRISM Climate Group 2013). All water balance 

components were computed independently for each land 

cover class within each HUC watershed and accumulated 

to estimate the totals for the watershed. For the NLCDbased impervious cover fraction, storage and ET were 

assumed to be negligible, and thus all PPT falling on the 

impervious portion of a watershed for a given month was 

assumed to generate surface runoff in the same month 

and was routed directly to the watershed outlet. No 

anthropogenic water use was included, and the model was 

run using off-the-shelf input datasets without calibration. 

Runoff Sensitivity to Changes in Climate

We quantified and compared changes in flow predictions 

at the study site under eight hypothetical climate change 

scenarios (table 4.2). Climate scenarios included: 

(1) increase PPT 10 percent, (2) decrease PPT 10 percent, 

(3) increase PPT 20 percent, (4) decrease PPT 20 percent, 

(5) increase TEMP 1 °C, (6) increase TEMP 2 °C, 

(7) increase PPT 10 percent and increase TEMP 2 °C, and 

(8) decrease PPT 10 percent and increase TEMP 2 °C. 

These climate change scenarios are hypothetical; however, 

they are reasonable projections of potential long-term 

changes in PPT and TEMP in the SEUS by the middle of 

the 21st century derived from General Circulation Models 

(Walsh and others 2014) or were designed to represent 

extremes in dry (e.g., 20 percent decrease in PPT) or 

wet (e.g., 20 percent increase in PPT) years. For each 

scenario, PPT and/or TEMP were adjusted uniformly by 

the specified amount at each time step. Predicted annual 

and seasonal flows for each scenario were compared to 

the baseline climate scenario to quantify changes in flow 

resulting from the hypothetical change in climate. The 

slope of the linear relation between the relative change 

in runoff and the relative change in PPT was computed, 

representing the sensitivity of runoff to changes in PPT or 

the climate elasticity of streamflow (Sankarasubramanian 

and Vogel 2001). This metric has been successfully used 

in several other assessments evaluating the influence of 

climate change on runoff (Jha and others 2006, Legesse 

and others 2010, Mengistu and Sorteberg 2012).

RESULTS

Predicted changes in mean annual runoff at the study site 

using the large-scale, uncalibrated WaSSI model were 

similar to those of the fine-scale, calibrated HSPF and 

WaterFALL® models under many of the hypothetical 

Table 4.2—Climate scenarios used to evaluate sensitivity 

of predicted runoff to climate change 

Scenarioa Identifier PPT change TEMP change

1 PPT +10% +10% No change

2 PPT -10% -10% No change

3 PPT +20% +20% No change

4 PPT -20% -20% No change

5 TEMP +1 No change +1 °C

6 TEMP +2 No change +2 °C

7 PPT +10%, TEMP +2 +10% +2 °C

8 PPT -10%, TEMP +2 -10% +2 °C

a

 For all scenarios, precipitation (PPT) and temperature (TEMP) were adjusted 

by the amount shown at each time step relative to the historical baseline period 

from 1980–1999. 

climate change scenarios considered (fig. 4.2). For 

example, under a climate scenario of a 10-percent 

reduction in PPT, the change in runoff among HSPF, 

WaterFALL®, and WaSSI was -18.8, -18.0, and -21.3 

percent, respectively. Under the more extreme changes 

in PPT (e.g., 20-percent decrease), WaSSI’s predicted 

change in runoff (-41.8 percent) was slightly higher than 

that predicted by HSPF (-36.0 percent) and WaterFALL®

(-34.5 percent). Predicted changes in runoff for scenarios 

of TEMP change were lower than those for changes in 

PPT, but HSPF, WaterFALL®, and WaSSI models generally 

agreed on the magnitude of change. For example, under 

TEMP changes of 2 °C, the change in runoff was -4.9, 

-7.2, and -8.0 percent among HSPF, WaterFALL®, and 

WaSSI, respectively. The SWAT model was generally 

more sensitive to changes in PPT and TEMP than the 

other models. 

The sensitivity of mean annual runoff to changes in PPT, 

or climate elasticity of streamflow, was similar among the 

HSPF, WaterFALL® and WaSSI models. The SWAT model, 

however; appeared to be more sensitive to both increases 

and decreases in PPT than the other models (fig. 4.3). 

Under increasing PPT scenarios, WaSSI (slope = 2.2) had 

similar sensitivity to that of HSPF (slope = 2.1), but both 

were slightly more sensitive than WaterFALL® (slope = 

1.9). Under decreasing PPT scenarios, WaSSI (slope = 

2.1) appeared to be somewhat more sensitive than HSPF 

(slope = 1.8) and WaterFALL® (1.7). 

Inspection of changes in monthly mean runoff as a result 

of a 20-percent decrease in PPT reveals that while changes 

in mean annual runoff were similar among some models, 

changes in monthly mean runoff were quite variable at 

this site (fig. 4.4). WaSSI and SWAT models were most 

sensitive to changes in PPT during the winter months 
-60

-40

-20

0

20

40

60

Change in runoff from baseline (%)

Scenario

HSPF

SWAT

WaterFALL®

WaSSI

PPT +10%

PPT -10%

PPT +20%

PPT -20%

TEMP +1

TEMP +2

PPT +10%, TEMP +2

PPT -10%, TEMP +2

Figure 4.2—Predicted relative change in mean annual runoff for the Flint River study site based on the 

eight hypothetical climate change scenarios. [See table 4.1 for hydrologic model definitions; see table 4.2 

for climate change scenario descriptions.]

-60

-40

-20

0

20

40

60

Change in runoff (%)

Change in precipitation (%)

HSPF

SWAT

WaterFALL®

WaSSI

-20 -10 0 10 20

1:1 line

USDA Forest Service, Southern Research Station 43 e-General Technical Report SRS-246

CHAPTER 4

Figure 4.3—Predicted change in mean annual runoff for various changes in 

precipitation. The slope of the linear relationship between change in runoff and 

change in precipitation represents the sensitivity of runoff to precipitation change. 

The dotted line represents the 1:1 line, and the slopes of the modeled output for 

HSPF, SWAT, WaterFALL®, and WaSSI would fall directly on the 1:1 line if the change 

in runoff were equal to the change in precipitation. [See table 4.1 for hydrologic 

model definitions.]
-60

-50

-40

-30

-20

-10

0

Relative change (%)

Month

HSPF

SWAT

WaterFALL®

WaSSI

(A)

1 2 3 4 5 6 7 8 9 10 11 12

-35

-30

-25

-20

-15

-10

-5

0

Absolute change (mm)

Month

HSPF

SWAT

WaterFALL®

WaSSI

(B)

1 2 3 4 5 6 7 8 9 10 11 12

44

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Figure 4.4—Relative (A) and absolute (B) changes in monthly mean runoff for a 20-percent precipitation 

decrease. [See table 4.1 for hydrologic model definitions.]

when runoff is highest, which likely explains why they are 

more sensitive to PPT decreases at the mean annual scale 

than HSPF or WaterFALL®. For example, mean relative 

changes in runoff in February were 47.3 and 52.6 percent 

for WaSSI and SWAT, respectively, while changes in 

February runoff for HSPF and WaterFALL® were 38 and 

31 percent, respectively. 

These differences in sensitivity of runoff to changes 

in PPT across months are related to the differences in 

seasonality of ET and soil moisture storage among models 

(fig. 4.5). While PPT was similar for all models, WaSSI 

and SWAT predicted higher ET rates during the winter 

months (November, December, and January) than did 

HSPF and WaterFALL®. Thus, reductions in PPT in the 

climate change scenarios during the winter months may 

impact runoff more than ET for WaSSI and SWAT because 

less excess water would be available to generate runoff 

due to the higher winter ET rates. WaterFALL® predicted 

the lowest winter ET rates, resulting in the highest winter 

runoff rates among models. 

DISCUSSION

Large- and fine-scale hydrologic models could be used in 

combination to identify specific areas of concern across 

large regions and to provide high-resolution quantitative 

estimates of changes in water supply ERFMs in these 

areas under changes in climate, but the differences in 

predicted streamflow response across large- and finescale models must be assessed before applying such a 

multi-scale modeling approach. The goal of this study 

was to examine the potential for combined application of 

large- and fine-scale hydrologic modeling approaches over 

large regions for climate change assessment studies. We 
HSPF SWAT WaSSI WaterFALL®

20

40

60

80

100

120

140

160

1 2 3 4567 8 9 10 11 12

Precipitation (mm)

Month

20

40

60

80

100

120

140

160

12345 6 7 8 9 10 11 12

Evapotranspiration (mm)

Month

0

10

20

30

40

50

60

70

80

1 2 3 4 5 6 7 8 9 10 11 12

Runoff (mm)

Month

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10 11 12

Soil moisture 

(% saturation)

Month

USDA Forest Service, Southern Research Station 45 e-General Technical Report SRS-246

CHAPTER 4

Figure 4.5—Monthly median precipitation, evapotranspiration, soil moisture, and runoff for all models under 

baseline conditions. [See table 4.1 for hydrologic model definitions.]

quantified differences in sensitivity of predicted runoff to 

climate change (i.e., PPT and TEMP) among one largescale (WaSSI) and three fine-scale (HSPF, SWAT, and 

WaterFALL®) hydrologic models. 

While the absolute magnitude of model-predicted runoff 

sensitivity to changes in PPT and TEMP may differ, the 

general pattern of the results of this study are consistent 

with previous work. Tang and others (2012) found that 

annual mean streamflow in the Salmon River Basin, ID, 

decreased 2–6 percent across sub-watersheds in the basin 

for a TEMP increase of 2 °C using the Variable Infiltration 

Capacity (VIC) model. Legesse and others (2010) used 

PRMS to evaluate the effect of changes in PPT and 

TEMP on runoff in the Meki River, Ethiopia, and found 

that runoff was more sensitive to increases in PPT (+80 

percent for +20 percent PPT) than decreases (-62 percent 

for -20 percent PPT), and a 1.5 °C increase in TEMP 

resulted in a 13-percent decrease in runoff. Mengistu and 

others (2012) used the SWAT model to evaluate changes 

in annual runoff in the Eastern Nile Basin and found that 

runoff increased 10–35 percent for a 10-percent increase 

in PPT, decreased 17–26 percent for a 10-percent decrease 

in PPT, and decreased up to 6 percent for a 2 °C increase 

in TEMP across three sub-watersheds in the basin. Jha and 

others (2006) used SWAT to evaluate sensitivity to PPT 

changes in the Upper Mississippi River Basin and found a 

26-percent reduction and 28-percent increase in runoff for 

10-percent decreases and increases in PPT, respectively.

Results of this study suggest that runoff predicted by largescale models (e.g., WaSSI) and fine-scale models (e.g., 

HSPF), despite differences in model structure and level of 

calibration, is similar in sensitivity to changes in PPT and 

TEMP at the annual scale (figs. 4.2 and 4.3). Annual runoff 

sensitivity for increases in PPT at the Flint River study site 

ranged from 1.9 to 2.2 among the HSPF, WaterFALL®, 

and WaSSI models, and sensitivity for decreases in PPT 

ranged from 1.8 to 2.1. These values are well within the 

range of streamflow elasticity to climate reported for the 

SEUS (Sankarasubramanian and Vogel 2001). Although 

there was generally good agreement in sensitivity of runoff 

to changes in PPT among models at the annual scale, there 

were differences in predicted sensitivity among models at 

the monthly scale (fig. 4.5). For example, under a scenario 

of a 20-percent decrease in PPT, the predicted change in 

March runoff was -33.9, -47.9, -28.9, and -42.0 percent for 

HSPF, SWAT, WaterFALL®, and WaSSI, respectively (fig. 

4.4a). The combination of differences in model structure 

(e.g., ET calculation method), parameterization (e.g., land 
46

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

cover characteristics), and calibration strategy evidently 

influences model predictions at the monthly scale. Further 

study is necessary to tease out the root cause of the 

differences in runoff sensitivity. While large- and finescale models have similar predictive performance at the 

monthly time step, in light of these findings caution should 

be used when combining large- and fine-scale models for 

regional resource management applications at time scales 

other than the long-term annual scale. 

LIMITATIONS

While this study provided unique insights into differences 

in predicted runoff sensitivity to climate among models, 

it is not a comprehensive assessment taking into account 

all of the factors that could influence these relationships. 

Some limitations of this study include:

1. Only a subset of the hydrologic models evaluated 

elsewhere in this report were considered in this 

study. Models included here consisted of those from 

government agencies and private organizations that 

were able to simulate the additional climate scenarios.

2. The models evaluated were not developed using 

the same calibration objective functions and input 

datasets. It is not known how these differences in 

model calibration strategy may affect predicted runoff 

sensitivity to changes in PPT and TEMP. 

3. Models evaluated were applied to a single basin in 

the SEUS region. The relative differences in predicted 

runoff sensitivity to changes in PPT and TEMP may 

vary in other hydroclimatic settings.

4. In general, hydrologic model streamflow predictions 

are subject to uncertainty in climate, land cover, soil, 

and LAI input data, as well as uncertainty in the 

representation of the physical processes that govern 

streamflow magnitude and timing. 

CONCLUSIONS

The objective of this study was to examine the potential 

for combining large- and fine-scale hydrologic modeling 

approaches over large regions for climate change 

assessment studies. We compared the sensitivity of runoff 

predictions from one large-scale and three fine-scale 

models to changes in PPT and TEMP and found that, while 

there were differences in model complexity and calibration 

strategy, predicted changes in runoff to changes in climate 

were similar at the long-term annual scale, but there were 

differences in predicted sensitivity among models at the 

monthly scale. Due to these seasonal differences, caution 

should be used when combining large- and fine-scale 

models for regional resource management applications 

at time scales other than the long-term annual scale. 

Future research is needed to understand the differences in 

predicted change in monthly runoff and to investigate the 

effect of model calibration strategy, as well as to evaluate 

differences in predicted runoff sensitivity across a larger 

set of gauged watersheds. Despite these limitations, the 

results of this study show promise in the potential to use 

a combination of large- and fine-scale models to examine 

climate change effects on regional water resources.
USDA Forest Service, Southern Research Station 47 e-General Technical Report SRS-246

CHAPTER 5

Using Regional-Scale Flow-Ecology Modeling to 

Identify Catchments Where Fish Assemblages are 

Most Vulnerable to Changes in Water Availability

INTRODUCTION

Environmental water studies over the last 2 decades have 

emphasized the inherent linkage and, in some cases, the 

tension between maintaining water for human use as 

well as for ecosystem needs (Acreman and others 2008, 

Kendy and others 2012, Poff and others 2010, Shenton and 

others 2012). Considerable emphasis has been placed on 

understanding the broad discharge patterns that influence 

the structural, functional, and life history strategies of 

biotic communities (Bunn and Arthinton 2002, Mims and 

Olden 2012, Naiman and others 2008). More recently, 

there has been an emphasis on developing hydrologic 

indices for characterizing the flow regime (Henriksen 

and others 2006, Monk and others 2007, Worrall and 

others 2014), systematically arranging streams and rivers 

into specific stream classes with respect to flow regime 

characteristics (Archfield and others 2013; Kennard and 

others 2010; Kennen and others 2007, 2009; McManamay 

and others 2012; Olden and Poff 2003), and building flowecology response models that link changes in streamflow 

and water availability to changes in assemblage structure 

and function (e.g., Arthington and others 2014; Chessman 

and others 2010; Freeman and others 2013; Kennen 

and others 2010, 2014; McManamay and others 2013; 

Stewart-Koster and others 2014; Turner and Stewardson 

2014). All of these studies emphasize the identification 

of the streamflow components needed to help determine 

ecological and environmental endpoints and the inherent 

linkages between changes in streamflow processes and 

ecosystem response. 

Changes in riparian and watershed-scale land use and 

associated alterations in stream habitat and streamflow 

processes have been linked to declines in native stream fish 

populations (Olden 2016) and a general downward trend in 

aquatic biodiversity across the globe (Dudgeon and others 

2006, Vörösmarty and others 2010). While minimizing 

impervious surfaces and maximizing the conservation of 

contiguous tracts of forested lands in watersheds support 

the preservation of stream fish populations (Kennen and 

others 2005), alterations in water availability, including 

impoundments, streamflow regulation, and water resource 

development, which are essential to meet the water needs 

of a growing population, are strongly linked to changes in 

native fish diversity, abundance, and resilience (Conroy 

and others 2003, Poff and Zimmerman 2010, Warren 

and others 2000). Confounding the effects of land use 

change and streamflow alteration are projected increases 

in drought frequency and duration associated with climate 

change (IPCC 2013, Melillo and others 2014), which can 

place further stress on water supplies and fish assemblage 

structure (Keaton and others 2005, Matthews and MarshMatthews 2003, White and others 2016). Understanding 

these linkages and the potential effect of changes in water 

availability on aquatic ecosystems is critical for longterm water management in areas facing significant water 

stress, especially when the needs of humans and aquatic 

ecosystems appear to conflict and sometimes result in 

legal proceedings. This is periodically the case in the 

Southeastern United States (SEUS) where water stress is 

known to occur in conjunction with drought cycles (e.g., 

Seager and others 2009). It is under these conditions that 

water managers must identify areas of concern and make 

informed decisions about water conservation that affect 

human and ecological use in these areas. Unfortunately, 

there is a lack of decision-support tools that identify 

areas of concern across broad regions, especially 

tools with a spatial resolution relevant to management 

decision making. 

The primary objective of this study was to demonstrate the 

efficacy of using relatively simple, large-scale hydrologic 

models in conjunction with ecological data to develop 

empirical flow-ecology response models that predict the 

effect of changes in water availability on fish species 

richness (FSR), an easily quantified assemblage metric. 

Additionally, we sought to use this modeling approach to 

identify catchments or “hot spots” of FSR change under 

a plausible set of future land use, climate, and withdrawal 

change scenarios and test the hypothesis that FSR in the 

North Carolina Piedmont will decrease with predicted 

increases in urban land use (i.e., impervious surfaces), 

changes in climate, and increases in water withdrawals 

(see Hain and others 2018).
48

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

METHODS

Study Area

We focused on catchments in the Piedmont ecoregion 

of North Carolina to develop empirical relationships 

between streamflow and FSR (fig. 5.1A). Seven major 

river basins are partially located within the Piedmont 

region. These include the Broad (BRD), Cape Fear 

(CPF), Catawba (CTB), Neuse (NEU), Roanoke (ROA), 

Tar-Pamlico (TAR), and Yadkin (YAD) River Basins 

(fig. 5.1B). The North Carolina Piedmont contains 

or intersects 886 12-digit Hydrological Unit Code 

catchments (hereafter HUC-12s) identified by the U.S. 

Department of Agriculture (USDA) Natural Resources 

Conservation Service’s (NRCS) Watershed Boundary 

Dataset Geographic Information System (GIS) layer. North 

Carolina has one of the highest rates of population growth 

in the United States and is now among the top 10 most 

populous States (U.S. Census Bureau 2016). Much of this 

growth has occurred in the Piedmont, which contains many 

of the State’s most rapidly growing cities (i.e., Charlotte, 

Durham, Raleigh, and Winston-Salem) (fig. 5.1A). 

The Piedmont region of North Carolina (fig. 5.1A) has 

a humid subtropical climate of warm summers and cool, 

moist winters. The region receives, on average, 107–117 

cm year-1 of precipitation (PPT) (North Carolina Climate 

Office 2016) and is made up of gently rolling forested 

hills, with elevations ranging from 60 to 470 m. The 

geology of the Piedmont is dominated by metamorphic 

(gneisses and schist) and igneous (granite, diorite, and 

gabbro) rocks overlain by “clayey” ultisols (soils with 

light upper layers and a reddish sub-soil) that were 

mainly formed through physical weathering and alluvial 

processes. These soils are rich in aluminum and silicates 

and contain eroded sediments mixed with organic material. 

Natural vegetative cover in this region consists mainly 

of mesic mixed hardwoods (e.g., American beech [Fagus 

grandifolia], tulip poplar [Liriodendron tulipifera], hickory 

[Carya spp.], and red and white oak [Quercus spp.]), 

though wetlands occur in some lower elevations, and 

patches of pine (Pinus spp.) forests are found in more xeric 

regions. Population growth and development in the North 

Carolina Piedmont have altered the natural landscape and 

increased water demand. Surface water and groundwater 

withdrawals have reduced baseflow, and estimates of water 

use show that, in 2010, the total gross fresh surface water 

withdrawals across the 54 counties in the region amounted 

to 11.7 billion m3

 year-1 (Maupin and others 2014).

Modeling Approach

For this study, we implemented a multi-step modeling 

approach (fig. 5.2). First, FSR was calculated for 385 

distinct fish sampling sites in the North Carolina Piedmont 

using data collected by the North Carolina Division of 

Water Resources (NCDWR). Average monthly streamflow 

for the sites was then predicted using the well-documented 

Water Supply Stress Index (WaSSI) model (Caldwell 

and others 2015). Fish species richness and streamflow 

predictions were used to build a boosted regression tree 

(BRT) flow-ecology model, which was used to predict 

the relationship between a subset of ecologically relevant 

flow metrics (ERFMs) and FSR in all HUC-12s in the 

North Carolina Piedmont. The results of the model were 

then used to predict FSR under three plausible scenarios 

of future water withdrawals, climate change, and increases 

in impervious surfaces. Finally, a “hot spot” analysis was 

used to identify individual HUC-12s that were most likely 

to be affected by changes in water availability.

Biological Data Aggregation

The Biological Assessment Unit (BAU) of the NCDWR 

began sampling each of the State’s 17 river basins on 

a 5-year rotation in 1990 (NCDENR 2006). Streams 

wadeable from shoreline to shoreline were sampled by 

the Stream Fish Community Assessment Program for an 

average distance of 183 m (600 feet). A four-person team 

collected all fish at each site using a modified two-pass 

depletion technique with two backpack electrofishing units 

and two persons netting. All fish were identified to species, 

enumerated, inspected for disease and deformities, and 

measured for total length before being released back into 

the stream. Specimens not easily identified in the field 

were preserved in 10-percent formalin and transported 

to the BAU laboratory in Raleigh, NC. Between 1990 

and 2012, 967 sampling events were performed by BAU 

at 385 unique sampling stations in the North Carolina 

Piedmont region. The sampling time window was limited 

to the spring between April and June which helped 

control for seasonal variability in flow conditions and was 

important for maintaining a consistent dataset for analysis 

across samples. Where unique stations were sampled 

multiple times, measures of FSR were averaged across all 

sampling events, resulting in a final sample size of 385 for 

developing flow-ecology relationships.

Upstream contributing catchments were delineated 

for each of the 385 unique NCDWR sampling stations 

(fig. 5.1B). Delineations were performed using Arc Hydro 

tools in ArcGIS 10.1 using digital elevation models 

procured from the North Carolina Floodplain Mapping 

Program (NC Floodplain Mapping Program 2013).

Streamflow Prediction

Streamflow was predicted for all study catchments using 

the WaSSI model, which was developed by the USDA 

Forest Service to assess the effects of climate change, 

land use change, and population growth on water supply 

stress, river flows, and aquatic ecosystems across the 

conterminous United States (Caldwell and others 2012, 
Counties

Cities

River basins

Study area

Ecoregion

Coastal Plain

Piedmont

Mountain

0 50 100 150 25

Kilometers

.

Fayetteville

Raleigh

Durham

Winston-Salem

High Point

Asheboro

Concord

Charlotte

Greensboro

BROAD

CATAWBA

YADKIN

ROANOKE

NEUSE

CAPE FEAR

TAR-PAMLICO

(A)

#

# #

#

#

# #

7 6

5

4

2 3

1

# USGS gauge

Gauged catchment

Fish species richness

<10.1

10.1–14.0 

14.1–18.0 

18.1–22.0 

>22.0

0 50 100 25

Kilometers

Broad

Catawba

Yadkin

Cape Fear

Neuse Tar-Pamlico

Roanoke

.

(B)

USDA Forest Service, Southern Research Station 49 e-General Technical Report SRS-246

CHAPTER 5

Figure 5.1—Map of North Carolina showing the study area spanning the Piedmont ecoregion, as well as parts of the 

Mountain and Coastal Plain ecoregions (A), and the range in FSR values for the BRT model training dataset in the North 

Carolina Piedmont and locations of seven USGS reference gauges used for the WaSSI model validation (B). Study area extent 

was chosen based on best professional judgement and discussions with NCDWR personnel (A). Each colored polygon in (B) 

represents the delineated contributing watershed for each ecological sample site.
Biological data 

aggregation

Streamflow 

prediction

Flow-ecology model 

development

Future scenarios

Projected climate 

change 2041–2060

Plausible water 

withdrawals

Projected 

impervious cover 

2060

Hot spot analysis

385 NCDWR fish sampling sites 

compiled for analysis

Identifying HUC-12s that are most 

susceptible to changes in water 

availability

Using boosted regression tree (BRT) 

modeling to relate FSR to streamflow 

metrics

Applying plausible scenarios of future 

changes in water availability

50

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Figure 5.2—Study modeling approach. Biological data were aggregated from NCDWR stream community assessment data. Streamflow was 

predicted using the WaSSI model.

2015, Sun and others 2011b). WaSSI has been successfully 

used in climate change assessments in the Eastern United 

States (Lockaby and others 2011, Marion and others 

2013, Sun and others 2013, Tavernia and others 2013) 

and for examining the nexus of water and energy at the 

national scale (Averyt and others 2011, 2013). WaSSI 

is an integrated monthly water balance and flow routing 

model that simulates the full hydrologic cycle for each of 

10 land cover classes at the HUC-12 scale. The 10 land 

cover classes are aggregated from the 17 classes of the 

2006 National Land Cover Database (NLCD) (Fry and 

others 2011). Infiltration, surface runoff, soil moisture, and 

baseflow processes for each HUC-12 catchment land cover 

class were computed using algorithms of the Sacramento 

Soil Moisture Accounting Model (SAC-SMA) (Burnash 

1995, Burnash and others 1973). State Soil Geographic 

(STATSGO) databases (USDA NRCS 2012) were used to 

compute the 11 SAC-SMA soil input parameters (Koren 

and others 2003). Monthly evapotranspiration (ET) was 

modeled with an empirical equation derived from multisite 

eddy covariance ET measurements (Sun and others 

2011a). Required data to estimate ET included monthly 

mean Moderate Resolution Imaging Spectroradiometer 

(MODIS) MOD15A2 leaf area index (LAI) (Zhao and 

others 2005), potential ET (PET) calculated as a function 

of air temperature (TEMP) and latitude (Hamon 1963), 

and PPT. This estimate of ET was then constrained by the 

soil water content computed by the SAC-SMA algorithm 

during extreme water-limited conditions. Monthly PPT 

and TEMP inputs were based on Parameter-elevation 

Relationships on Independent Slopes Model (PRISM) 

estimates (PRISM Climate Group 2013). All water balance 

components were computed independently for each land 

cover class within each catchment and accumulated to 

estimate the totals for the catchment. For the NLCD-based 

impervious cover fraction, storage and ET were assumed 

to be negligible, and thus all PPT falling on the impervious 

portion of a catchment for a given month was assumed to 

generate surface runoff in the same month and was routed 

directly to the catchment outlet. 

Although the WaSSI model can be calibrated, no 

calibration of model input parameters was performed 

for this study. WaSSI was developed to include the 

key ecohydrological processes that affect the water 

balance with off-the-shelf input datasets while having 

an acceptable level of predictive performance without 

calibration. In doing so, the model is not subject to the 

complexities and uncertainties associated with transferring 

model parameters from calibrated to ungauged catchments 

(Siviplan and others 2003) and using the model to assess 

the effect of climate or land cover scenarios outside of 

the conditions for which the model is calibrated. Despite 

being uncalibrated, WaSSI has been found to have 

similar predictive performance at the monthly time step 

to other calibrated, process-based models (Caldwell and 

others 2015). 

Streamflow was predicted using the monthly WaSSI output 

from the years 1991–2010. This output represented the 

average streamflow for each month of the year over the 20-

year period and resulted in 12 average monthly streamflow 
USDA Forest Service, Southern Research Station 51 e-General Technical Report SRS-246

CHAPTER 5

values for each site. We calculated a suite of ERFMs from 

these monthly averages targeting flow metrics that account 

for the magnitude and seasonality (timing) of streamflow 

(Poff and others 1997) in an effort to capture flow signals 

important to fish life history. This included minimum 

monthly streamflow in m3 s-1 (low flow; MinCMS); 

maximum monthly streamflow (high flow; MaxCMS); 

average monthly streamflow (average flow; AveCMS); 

average streamflow for April, May, and June (spring flow; 

AMJCMS); average streamflow for August (summer flow; 

AugCMS); and a measure of the coefficient of monthly 

streamflow variability (FloVar). FloVar was computed by 

dividing the standard deviation of the decreased monthly 

flow values by the original monthly mean streamflow. 

This measure of streamflow variability is equivalent to 

“MA39,” a common measure of monthly streamflow 

variability presented in Olden and Poff (2003) and 

Henriksen and others (2006). Additionally, the proportion 

of surface flow coming from impervious surfaces 

(QFRAC_IMP) was estimated for each catchment. All 

statistics were estimated for contributing catchments to 

each NCDWR sample site, as well as all HUC-12s in the 

North Carolina Piedmont. The monthly time step allows us 

to evaluate the ability of large-scale, easily parameterized 

models to provide flow information that is useful for 

determining changes in ecosystem integrity.

We validated the WaSSI model flow predictions by 

computing classical hydrologic model fit statistics as 

well as the prediction of ERFMs at seven USGS gauges 

located in the study area (table 5.1, fig. 5.1B). Gauges 

identified as reference sites in the USGS Gages II database 

(Falcone 2011, Falcone and others 2010) were selected 

for validation procedures and were either co-located 

with a fish training site or had one or more fish training 

sites anywhere within the gauged basin. Classical fit 

statistics evaluated included bias in mean streamflow, 

the Nash-Sutcliffe Efficiency (NSE) statistic (Nash and 

Sutcliffe 1970), the root mean squared error (RMSE), 

and the coefficient of determination (R2

). Ecologically 

relevant flow metrics evaluated included MaxCMS and 

FloVar that were used in the four-variable flow-ecology 

model (described below). Bias in mean streamflow within 

±25, ±15, and ±10 percent was considered indicative 

of satisfactory, good, and very good hydrologic model 

performance, respectively (Moriasi and others 2007). 

Similarly, NSE values that are >0.50, >0.65, and >0.75 

for prediction of monthly streamflow were considered to 

be indicative of satisfactory, good, and very good model 

performance, respectively (Moriasi and others 2007). The 

NSE can range from negative infinity to 1.0; the closer 

NSE is to 1.0, the better the model fit. Negative values 

of NSE indicate that using the mean of the observations 

provides a better fit than the model. A hydrologic 

uncertainty of ±30 percent was used to aid in placing 

model prediction bias of ERFMs into context with inherent 

variability in streamflow and flow measurement (Murphy 

and others 2013). 

Table 5.1—Summary of classical hydrological model fit statistics and bias in prediction of the ecologically 

relevant flow metricsa

 (ERFMs) used in the four-variable boosted regression tree (BRT) model across the seven 

USGS gauges used for WaSSI model validation 

Classical model fit statistics ERFMs

Site Gauge Description

Drainage 

area

(km2 )

Bias in 

mean 

streamflow

(%) NSE

RMSE

(m3

 s-1

[cms]) R2

Bias in 

MaxCMS

(%)

Bias in 

FloVar

(%)

1 02077200 Hyco Creek Near Leasburg, NC 121.7 16 0.60 0.85 0.63 -12 -26

2 02081500 Tar River near Tar River, NC 428.4 16 0.74 2.05 0.76 -17 -34

3 02082950 Little Fishing Creek near White Oak, NC 460.9 10 0.78 2.01 0.79 -17 -44

4 02112360 Mitchell River near State Road, NC 205.3 -10 0.50 1.17 0.70 2.4 73

5 02118500 Hunting Creek near Harmony, NC 400.5 13 0.60 2.30 0.69 3.4 3.7

6 02125000 Big Bear Creek near Richfield, NC 144.5 6.9 0.81 0.76 0.81 -11 -18

7 02128000 Little River near Star, NC 273.5 -8.3 0.71 1.45 0.72 -16 0.8

Mean 

(standard deviation)

290.7 

(140.0)

6.2 

(11.0)

0.68 

(0.11)

1.51 

(0.62)

0.73 

(0.06)

-9.5

 (8.9)

-6.4 

(39.1)

a

 Bias in mean streamflow within ±25, ±15, and ±10 percent is considered indicative of satisfactory, good, and very good hydrological model performance, 

respectively, while Nash-Sutcliffe efficiency (NSE) values that are >0.50, >0.65, and >0.75 for prediction of monthly streamflow are considered to be 

indicative of satisfactory, good, and very good model performance, respectively. A hydrologic uncertainty of ±30 percent was used to aid in placing model 

prediction bias of ERFMs into context with inherent variability in streamflow and flow measurement (Murphy and others 2013). 

FloVar = coefficient of monthly streamflow variability; MaxCMS = maximum monthly streamflow in m3 s-1 (cms); NSE = Nash-Sutcliffe Efficiency; R2 = 

coefficient of determination; RMSE = root mean squared error.
52

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Flow-Ecology Model Development

We developed a FSR BRT model using observed 

biological data and WaSSI streamflow predictions for 

all 385 NCDWR sample sites in the training dataset. 

Boosted regression tree models are only briefly described 

here as their use and technical details (e.g., Breiman 

and others 1984, De’ath and Fabricius 2000, Prasad and 

others 2006) as well as application (Aertsen and others 

2010; Brown and others 2012; Clapcott and others 2011; 

Elith and others 2008; Leclere and others 2011; Waite 

and others 2010, 2012) have been widely presented in 

the literature. Boosted regression trees are part of the 

classification and regression tree (CART) or decision tree 

family, a family of techniques used to advance single 

classification or regression trees by averaging the results 

for each binary split from numerous trees or forests. The 

objective of BRT models is to reduce the predictive error 

and improve overall performance (De’ath 2007, Elith 

and others 2008). In BRT, after the initial tree has been 

developed, successive trees are grown on reweighted 

versions of the data, giving more weight to cases that 

are incorrectly classified than those that are correctly 

classified within each growth sequence (Waite and others 

2012). As more and more trees are grown in BRT, the 

large number of trees increases the chance that cases that 

are difficult to classify initially are correctly classified, 

thus representing an improvement to the basic averaging 

algorithm used in random forest (De’ath 2007). Boosted 

trees and random forest models retain the positive aspects 

of single trees seen in CART models, yet have improved 

predictive performance, can easily assess nonlinearities 

and interactions, and can provide an ordered list of the 

importance of the explanatory variables (De’ath 2007, 

Leclere and others 2011). 

Although BRT offers improved modeling performance 

over CART, the simple single tree obtained from CART 

is lost, making visualization of the results more difficult. 

Partial dependency plots (PDPs) provide a way to visualize 

the effect of a specific explanatory variable on the response 

variable after accounting for the average effects of all 

other explanatory variables (De’ath 2007, Elith and others 

2008); PDPs for selected variables important in models 

appear as examples in the results. Boosted regression tree 

models were run using the gbm library in R and specific 

code from Elith and others (2008). 

Boosted regression tree models were developed using 

FSR as the response variable and ERFMs and river basin 

as explanatory variables. We developed the BRT models 

using the training dataset (fig. 5.1B) with a bag fraction 

of 0.5, a learning rate of 0.004, and a tree complexity 

of 3. A bag fraction of 0.5 indicates that each tree is 

developed using a random selection of 50 percent of the 

data. The learning rate influences the total number of trees 

evaluated for a model, while tree complexity controls 

whether interactions are fitted, with a value of 3 allowing 

the assessment of up to 3-way interactions. Variable 

relative importance (VRI) was calculated using formulae 

developed by Friedman (2001) and implemented in the R 

gbm library to estimate the relative importance of predictor 

variables (Waite and others 2012). Calculations of VRI 

are based on the number of times a variable is selected 

for splitting, weighted by the squared improvement to the 

models as a result of each split, averaged over all trees. 

The relative importance of each variable is scaled so that 

the sum adds to 100, with higher numbers indicating 

stronger influence on the modeled response. Due to the 

size of the training dataset, we implemented a k-fold crossvalidation technique using the R function gbm.step. The 

k-fold cross-validation splits the dataset into k partitions, 

keeping one partition for testing and the remaining 

partitions for fitting the model (Hastie and others 2009). 

This technique generally has low bias, and the predictive 

performance of a k-fold cross-validation and validation 

using an independent dataset is highly similar (Elith and 

others 2008). Additionally, k-fold cross-validation is 

known to provide a computational advantage over leaveone-out techniques and provides a more accurate estimate 

of the test error rate (James and others 2013). Goodness of 

fit was measured using the equivalent R2

, estimated as: 

(TD - RD) / TD

where 

 TD = total deviance 

 RD = residual deviance 

Initially we developed an eight-variable BRT model 

using the primary subset of ERFMs outlined above. 

Boosted regression tree approaches have been shown 

to overfit models (Aertsen and others 2010, Elith and 

others 2008). Therefore, we developed a reduced-variable 

model using only those variables identified as having a 

relative importance >10 percent (fig. 5.3). The final model 

variables were selected after evaluating a Spearman rank 

correlation matrix of explanatory variables (table 5.2) 

and the effects on model fit (i.e., equivalent R2

), and by 

examining the PDPs of all eight explanatory variables (not 

shown). This secondary evaluation allowed us to reduce 

the number of explanatory variables from eight to four 

without a loss of variability accounted for by the BRT 

response model. The final reduced four-variable model 

identified those variables most critical for assessing the 

effects of climate, streamflow, and land use changes on 

FSR and insured a high level of parsimony for use in 

future management scenarios. We used the final reduced 
Basin

QFRAC_IMP

FloVar

MaxCMS

MinCMS

AugCMS

AMJCMS

AveCMS

Relative importance

(A)

0 4 8 12 16 20 24 28 32

Basin

QFRAC_IMP

FloVar

MaxCMS

MinCMS

AugCMS

AMJCMS

AveCMS

Relative importance

(A)

0 4 8 12 16 20 24 28 32

Basin

MaxCMS

QFRAC_IMP

FloVar

Relative importance

(B)

0 4 8 12 16 20 24 28 32

Basin

MaxCMS

QFRAC_IMP

FloVar

Relative importance

(B)

0 4 8 12 16 20 24 28 32

USDA Forest Service, Southern Research Station 53 e-General Technical Report SRS-246

CHAPTER 5

Figure 5.3—Summary of the relative importance of the predictor variables included in (A) the full eight-variable (equivalent R2 = 

0.47) and (B) four-variable (equivalent R2 = 0.50) boosted regression tree models. [MinCMS = minimum monthly streamflow in m3

 s-1; 

MaxCMS = maximum monthly streamflow; AveCMS = average monthly streamflow; AMJCMS = average streamflow for April, May, 

and June (spring flow); AugCMS = average streamflow for August (summer flow); FloVar = measure of the coefficient of monthly 

streamflow variability; QFRAC_IMP = proportion of surface flow coming from impervious surfaces; Basin = the specific river basin in 

which the training site is located.]

Table 5.2—Spearman correlations and p-values (in parentheses)a of predictor variables for the boosted 

regression tree (BRT) models across the 385 North Carolina Division of Water Resources (NCDWR) 

sample sites 

Variable Basin AveCMS MinCMS MaxCMS AMJCMS AugCMS FloVar QFRAC_IMP FSR

Basin 0.07 

(0.1802)

0.01 

(0.8556)

0.11 

(0.0258)

0.04 

(0.4061)

0.02 

(0.6536)

0.11 

(0.0310)

-0.15 

(0.0025)

0.1 

(0.0508)

AveCMS 0.96 

(<0.0001)

0.99 

(<0.0001)

0.99 

(<0.0001)

0.94 

(<0.0001)

-0.22 

(<0.0001)

0.05 

(0.2893)

0.2 

(<0.0001)

MinCMS 0.9 

(<0.0001)

0.96 

(<0.0001)

0.99 

(<0.0001)

-0.42 

(<0.0001)

0.11 

(0.0359)

0.13 

(0.0092)

MaxCMS 0.96 

(<0.0001)

0.88 

(<0.0001)

-0.09 

(0.0869)

0.01 

(0.8612)

0.22 

(<0.0001)

AMJCMS 0.94 

(<0.0001)

-0.29 

(<0.0001)

0.03 

(0.6049)

0.18 

(0.0004)

AugCMS -0.42 

(<0.0001)

0.13 

(0.0128)

0.12 

(0.0147)

FloVar -0.21 

(<0.0001) 

0.2 

(<0.0001)

QFRAC_IMP -0.21 

(<0.0001)

FSR

a

 Bolded p-values are significant (α <0.05). Bolded variables represent those retained in the four-variable BRT model. 

AveCMS = average monthly streamflow; MinCMS = minimum monthly streamflow; MaxCMS = maximum monthly streamflow; AMJCMS = average 

streamflow for April, May, and June (spring flow); AugCMS = average streamflow for August (summer flow); FloVar = measure of the coefficient of 

monthly streamflow variability; QFRAC_IMP = proportion of surface flow coming from impervious surfaces; FSR = fish species richness; QFRAC_

IMP = the proportion of surface flow coming from impervious surfaces.
54

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

four-variable BRT model to predict FSR for all 385 

NCDWR sample sites and regressed observed FSR against 

predicted FSR with a 95-percent confidence interval 

(Piñeiro and others 2008).

Scenarios

The final reduced four-variable model developed for 

the training data was then used to predict FSR in each 

HUC-12 in the North Carolina Piedmont under current 

conditions as well as three future scenarios. These 

scenarios included (1) projected average climate for 

the years 2041–2060, (2) impervious cover projections 

for the year 2060, and (3) plausible water withdrawals 

from each HUC-12. For climate projections, statistically 

downscaled 1/8- x 1/8-degree (~12 x 12 km) 1961–2099 

monthly PPT and TEMP predicted by the National 

Oceanic and Atmospheric Administration’s Geophysical 

Fluid Dynamics Laboratory coupled climate model CM2.0 

for the A2 growth and emission scenario was obtained 

from the World Climate Research Programme Coupled 

Model Intercomparison Project Phase 3 (CMIP3) dataset 

(Meehl and others 2007). The Intergovernmental Panel 

on Climate Change (IPCC) Special Report on Emissions 

Scenarios (SRES) (Nakicenovic and others 2000) 

characterized the A2 storyline as a very heterogeneous 

world with continuously increasing global population and 

regionally oriented economic growth with relatively slow 

technological change. The A2 (High) SRES scenario was 

selected because it represents potentially the most likely 

emission scenario as post-2000 global carbon emissions 

estimates indicate that current emissions are tracking the 

higher of the SRES emission projections (Raupach and 

others 2007). The CM2.0 climate model was selected 

because it represents a “mid-range” scenario among the 16 

climate models evaluated in CMIP3 for the United States 

(Treasure and others 2014). Average monthly PPT and 

TEMP predictions were estimated for each HUC-12 using 

area-weighted means. Predicted FSR for the 2041 to 2060 

time period was compared to the 1991 to 2010 time period 

to evaluate potential climate change effects.

Impervious surface projections for the year 2060 

were derived from the U.S. Environmental Protection 

Agency’s Integrated Climate and Land-Use Scenarios 

(ICLUS) project for the A2 growth and emission scenario 

(Bierwagen and others 2010, USEPA 2009) to match the 

climate change scenario. The ICLUS project develops 

future impervious surface scenarios that are “broadly 

consistent with global-scale, peer-reviewed storylines of 

population growth and economic development” (USEPA 

2009). Projections are based on regression models that 

relate the 2001 NLCD impervious surface database with 

housing density estimates (a derivative of demographic 

projections), which enables forecasting likely changes 

under SRES growth scenarios (scenario A2 was used for 

this project) (USEPA 2009). Impervious cover effects 

were assessed by comparing FSR under projected 2060 

impervious cover to that of 2006 using the baseline climate 

data from 1991 to 2010.

The North Carolina Ecological Flow Science Advisory 

Board (NCEFSAB) recommended a “flow-by” criteria 

where ecological flows should be 80–90 percent of the 

instantaneous modeled baseline flow (NCEFSAB 2013). 

Consistent with this recommendation, we modeled 

the effect of reduced flows on FSR by systematically 

decreasing the amount of total streamflow predicted for 

each HUC-12 in the North Carolina Piedmont from 5–25 

percent at 5-percent intervals (i.e., a 95- to 75-percent 

flow-by), thereby bracketing the range recommended 

by the NCEFSAB. We then used these decreased flow 

values as explanatory variables in our BRT prediction 

models. Welch’s t-tests (Welch 1947) were performed 

using the R function t.test to test for differences between 

mean expected values for current conditions and values 

predicted under future scenarios (water withdrawals, 

impervious surface projections, and climate change). A 

t-test tests the hypothesis that predictions for future and 

withdrawal scenarios are equal to each other. The Welch’s 

modification adjusts the degrees of freedom for predictions 

whose variances are not equal (Welch 1947). Significant 

(α <0.05) p-values indicate that predictions are not equal.

RESULTS

The WaSSI model reasonably captured the magnitude 

and variability in observed flows at the seven validation 

sites within the study region (fig. 5.4, table 5.1). Model 

performance was satisfactory or better at all sites evaluated 

for classical model fit statistics. Absolute bias in mean flow 

was satisfactory (within ±25 percent) at two sites, good 

(within ±15 percent) at three sites, and very good (within 

±10 percent) at two sites. The NSE was near-satisfactory 

at one site (0.50), satisfactory (>0.50) at two sites, good 

(>0.65) at two sites, and very good (>0.75) at two sites. 

The mean bias in mean flow across all sites was +6.2 

percent (mean absolute bias 11.5 percent), while the mean 

NSE was 0.68; both statistics reflected good performance 

overall. Similarly, bias in the MaxCMS and FloVar 

ERFMs used in the four-variable flow-ecology model 

was generally within the range of hydrologic uncertainty 

(within ±30 percent). Bias for four of the seven sites 

was within ±30 percent for FloVar and was within ±30 

percent at all sites for MaxCMS. Sites 2, 3, and 4 FloVar 

bias was not within ±30 percent; however, site 2 was only 

marginally outside the range of hydrologic uncertainty 

at -34 percent. FloVar bias was greatest for site 4 at 73 

percent. Comments on flow modification in the USGS 

Gages II database indicate that there are small reservoirs in 

the headwaters of this catchment (Falcone 2011, Falcone 

and others 2010), possibly supplementing flows during 
Site 1: Gage 02077200 Hyco Creek near Leasburg, NC

Mean monthly streamow (m3 s-1)

0

2

4

6

8

10

Site 2: Gage 02081500 Tar River near Tar River, NC

0

5

10

15

20

25

30

35

Site 3: Gage 02082950 Little Fishing Creek near White Oak, NC

0

10

20

30

40

50

60

Site 4: Gage 02112360 Mitchell River near State Road, NC

0

2

4

6

8

10

12

Site 5: Gage 02118500 Hunting Creek near Harmony, NC

0

5

10

15

20

25

Site 6: Gage 02125000 Big Bear Creek near Richeld, NC

0

2

4

6

8

10

12

Site 7: Gage 02128000 Little River near Star, NC

1/1990 1/1992 1/1994 1/1996 1/1998 1/2000 1/2002 1/2004 1/2006 1/2008 1/2010

0

5

10

15

20

25

USDA Forest Service, Southern Research Station 55 e-General Technical Report SRS-246

CHAPTER 5

Figure 5.4—Observed (circles) and predicted (lines) mean monthly streamflow hydrographs for the seven USGS reference 

gauges used for WaSSI model validation.
56

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

dry periods and reducing FloVar in the observed time 

series relative to natural conditions simulated by WaSSI 

(fig. 5.4). Indeed, FloVar was lowest at site 4 among all 

sites, and thus small absolute differences in FloVar result 

in large relative differences. Overall, the validation results 

indicate that our hydrologic modeling approach provides 

reasonable approximations of ERFMs for flow-ecology 

modeling that fall within commonly applied bounds of 

uncertainty and bias.

Observed FSR values in the North Carolina Piedmont 

ranged from 5.0 to 28.5 across the 385 sites (fig. 5.1B, 

table 5.3), with an average of 16.4. Among river basins, 

the TAR, ROA, and NEU had the highest average FSR 

values, whereas the BRD, YAD, CPF, and CTB had 

slightly lower averages (table 5.3). By comparison, the 

highest MaxCMS values were found in the TAR, BRD, 

and NEU, while the CTB, ROA, YAD, and CPF were 

slightly lower (table 5.3). In addition to having the lowest 

MaxCMS values, the CPF had the highest QFRAC_IMP 

values along with the NEU, followed by the CTB, YAD, 

BRD, TAR, and ROA (table 5.3). 

The explanatory variables in the reduced four-variable 

model (all variables with relative importance <10 percent 

removed) consisted of Basin, QFRAC_IMP, MaxCMS, 

and FloVar, with an estimated equivalent R2 = 0.50 

(table 5.4). Basin, a categorical variable, was the most 

influential variable in the model (31.3-percent relative 

importance), followed by MaxCMS (23.7 percent), 

QFRAC_IMP (23.1 percent), and FloVar (21.9 percent) 

(fig. 5.3B). All flow variables show distinct relations 

with the fitted values (fig. 5.5). Basin accounts for basinspecific factors such as geology, topography, habitat, 

and latitude, so it is not surprising that Basin had a 

significant influence on the model outcome. Although each 

variable exhibits some variability, the overall response 

pattern indicates a negative response between FSR and 

QFRAC_IMP and a positive response between FSR and 

both MaxCMS and FloVar, although this response may 

not be linear (fig. 5.5). Further, the PDP for QFRAC_IMP 

shows a fairly rapid linear decline in FSR at relatively low 

levels of surface flow coming from impervious surfaces 

(fig. 5.5). Conversely, there is a strong increase in FSR 

between 3 and 7 m3

 s-1 in the PDP for MaxCMS. The 

interactions between MaxCMS, QFRAC_IMP, and FloVar 

indicate that FSR is highest when MaxCMS and FloVar 

are high, but QFRAC_IMP is low (fig. 5.6). Across the 

ranges of MaxCMS and FloVar values, FSR remains quite 

low when QFRAC_IMP is high (fig. 5.6). The slope of the 

regression for observed FSR values versus those predicted 

by the BRT model was 0.41 (predicted FSR = 9.73 + 

0.41 × observed FSR) with an adjusted R2 value of 0.48, 

indicating a relatively good predictive fit with only slight 

bias across the range of values. 

The northeastern part of the Piedmont including the 

ROA, TAR, and NEU river basins had higher FSR 

values than the rest of the region under the baseline 

scenario as illustrated in figures 5.5 and 5.7A. Projected 

climate change by 2041–2060 increased FSR by 0.35 

species (table 5.4) on average (p = 0.0042), ranging from 

a decrease of 2.19 to an increase of 3.10 (fig. 5.7B). 

Projected changes in impervious cover resulted in an 

insignificant decrease (p = 0.1817) in FSR of 0.16 species 

across the region on average (table 5.4). Fish species 

richness decreased significantly across the region with 

increasing water withdrawals (table 5.4; figs. 5.7D 

and 5.8). Under the water withdrawal scenarios, a 

significant loss in FSR of 0.49 species was predicted 

with a 15-percent reduction in flow (p = 0.0001), while a 

reduction in flow of 25 percent was predicted to have an 

average loss of one species (p <0.0001) (table 5.4). 

Table 5.3—Predicted mean (standard deviation, in parentheses) 

ecologically relevant flow metrics (ERFMs) and fish species richness 

(FSR) across the 385 North Carolina Division of Water Resources sample 

sites for the Broad (BRD), Cape Fear (CPF), Catawba (CTB), Neuse (NEU), 

Roanoke (ROA), Tar-Pamlico (TAR), and Yadkin (YAD) River Basins

BRD CPF CTB NEU ROA TAR YAD

All 

basins

FSR 15.63 

(2.8)

15.48 

(4.36)

13.59 

(3.63)

18.95 

(3.82)

19.36 

(3.37)

19.91 

(3.65)

15.56 

(3.95)

16.35 

(4.32)

MaxCMS 2.8 

(3.89)

1.43 

(1.68)

1.76 

(1.15)

2.46 

(2.06)

1.79 

(1.39)

3.07 

(3.68)

1.75 

(1.17)

1.95 

(2.05)

QFRAC_IMP 0.08 

(0.07)

0.21 

(0.18)

0.2 

(0.17)

0.21 

(0.14)

0.05 

(0.04)

0.06 

(0.05)

0.12 

(0.12)

0.14 

(0.15)

FloVar 0.75 

(0.04)

0.81 

(0.12)

0.74 

(0.09)

0.78 

(0.08)

0.8 

(0.1)

0.85 

(0.06)

0.83 

(0.16)

0.8 

(0.12)

MaxCMS = maximum monthly streamflow; QFRAC_IMP = the proportion of surface flow coming 

from impervious surfaces; FloVar = measure of the coefficient of monthly streamflow variability.
−3 −2 −1 0 1 2

BRD CPF CTB NEU ROA TAR YAD

Basin

Fitted function

0.0 0.1 0.2 0.3 0.4 0.5 0.6

−3 −2 −1 0 1 2

QFRAC_IMP

Fitted function

0 5 10 15

−3 −2 −1 0 1 2

MaxCMS

Fitted function

0.6 0.7 0.8 0.9 1.0 1.1 1.2

−3 −2 −1 0 1 2

FloVar

Fitted function

USDA Forest Service, Southern Research Station 57 e-General Technical Report SRS-246

CHAPTER 5

Table 5.4—Boosted regression tree predictions for the four-variable model across the 

North Carolina Piedmont ecoregion 

Climate 

change

Impervious 

cover Change in flow (water withdrawal)c

Four-variable model 

(equivalent R2

 = 0.50) 2041–2060b 2041–2060b -5% -10% -15% -20% -25%

t-test p-valuea 0.0042 0.1817 0.7909 0.1047 0.0001 <0.0001 <0.0001

Mean expected 18.04 18.04 18.04 18.04 18.04 18.04 18.04

Mean predicted 18.39 17.88 18.012 17.84 17.55 17.23 17.05

Mean change 

(predicted - expected)

0.35 -0.16 -0.03 -0.20 -0.49 -0.81 -0.99

a

 Welch’s t-tests were performed to test for differences between mean expected values for current conditions and values 

predicted under future scenarios (water withdrawals, impervious surface projections, and climate change). Bolded values 

represent a significant difference (p <0.05) between current conditions and predicted values. 

b

 2041–2060 represents the time period for the climate change and impervious surface scenarios. 

c

 Percentage values from -5 to -25 represent water withdrawal scenarios. 

R2 = coefficient of determination.

Figure 5.5—Partial dependency plots for variables in the final four-variable boosted regression tree model for FSR. 

Boosted regression tree partial dependency plots show the response form of FSR (y-axis = fitted function of FSR) based 

on the effect of individual explanatory variables with the response of all other variables removed. [FloVar = measure of 

the coefficient of monthly streamflow variability; MaxCMS = maximum monthly streamflow; QFRAC_IMP = the proportion 

of surface flow coming from impervious surfaces.]
QFRAC_IMP

0.0

0.1

0.2

0.3

0.4

0.5 MaxCMS

0.6 0

5

10

15

20

Fitted value

10

12

14

16

18

20

22

QFRAC_IMP

0.0

0.1

0.2

0.3

0.4

0.5 FloVar

0.6 0.0

0.2

0.4

0.6

0.8

1.0

1.2

Fitted value

10

12

14

16

18

20

22

FloVar

0.0

0.2

0.4

0.6

0.8

1.0 MaxCMS

1.2 0

5

10

15

20

Fitted value

10

12

14

16

18

20

22

58

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Figure 5.6—Interaction plots of QFRAC_IMP, MaxCMS, and FloVar, the three continuous variables in the four-variable model. The 

y-axis fitted value represents the effect of the interaction on the response variable FSR. [FloVar = measure of the coefficient of 

monthly streamflow variability; MaxCMS = maximum monthly streamflow; QFRAC_IMP = the proportion of surface flow coming from 

impervious surfaces.]

Under each future scenario, some HUC-12s were more 

likely to experience changes in FSR than the average 

HUC-12 (figs. 5.7 and 5.8). These results indicated that 

some HUC-12s will lose species more quickly than 

the region average. The hot spots shown in figure 5.8 

(HUC-12s that could potentially lose more than one fish 

species under the 5-, 10-, 15-, and 20-percent withdrawal 

scenarios) identify the catchments that are most vulnerable 

to a loss in FSR and provide managers with a mechanism 

for prioritizing catchments that are most susceptible to 

changing water availability. The percentage of HUC-12s 

predicted to experience a decrease in FSR varied across 

river basins, with FSR in some basins (CPF, CTB, TAR, 

and YAD) appearing to be particularly sensitive to changes 

in ERFMs (table 5.5). Predicted FSR decreased in the 

majority of HUC-12s under all flow reduction scenarios 

(table 5.5). Even under the climate change scenario, 

where average FSR across all HUC-12s was predicted 

to increase, a large percentage (33 percent) of HUC-12s 

were predicted to decrease in FSR (table 5.5). Further, the 

climate change scenario showed a large increase in mean 

MaxCMS (3.32–15.28 m3

 s-1) across all major river basins 

(table 5.6), resulting in a mean increase in FSR. MaxCMS 

increased in some catchments under the impervious cover 

scenario; however, QFRAC_IMP also increased (table 5.6) 

resulting in a net decrease in FSR (table 5.4).

DISCUSSION

In this study we evaluated whether relatively simple 

regional-scale hydrologic models can be used in 

conjunction with ecological data to develop empirical 

flow-ecology response models that predict the effect of 

changes in water availability on FSR at a spatial scale 

relevant to management. We also sought to use the 

empirical flow-ecology models to identify “hot spots” 

of FSR change under plausible scenarios representing 

changes in water withdrawals (e.g., 5–25 percent), land 

use (derived from known build-out scenarios), and 

climate (the A2 high emission scenario) at the HUC12 level. We postulated that a decline in predicted FSR 

would be attributable to changes in climate and increases 

in impervious surfaces and water withdrawals. Our 

findings indicate that changes in streamflow associated 

with plausible future water withdrawals may result in a 

significant loss in FSR, and for the withdrawal scenarios 

across the region as a whole, losses appear to be directly 

linked to the quantity of water withdrawn. Although the 

future impervious cover scenario was not found to be 

significant across the region as a whole, decreases in FSR 

of one or more species were predicted in many HUC-12s 

proximal to the highly urban regions of North Carolina 

including Charlotte, Greensboro, Raleigh-Durham, and 

Fayetteville (fig. 5.7C). Under the climate change scenario, 

FSR was actually predicted to increase significantly across 

all HUC-12s. While this was contrary to our hypothesis, 

there were many individual HUC-12s where FSR was 

predicted to decrease (table 5.5). 

The key variable driving the average increase in FSR 

for the climate change scenario appears to be MaxCMS 

(maximum monthly streamflow for the 20-year period 

of record). Under this scenario, MaxCMS was the only 

variable from the four-variable BRT model to change 

substantially from the 1991–2010 average. MaxCMS was 

highly correlated with FSR in the training data, so these 

results should not be surprising. However, these findings 

could indicate a link between predicted changes in climate, 

maximum monthly or seasonal flows in river systems, 
!( !( !( !( !( !( !( !( !(

Predicted FSR, 1991–2010

<12.1

12.1 – 13.0

13.1 – 14.0

14.1 – 15.0

15.1 – 16.0

16.1 – 17.0

17.1 – 18.0

18.1 – 19.0

19.1 – 20.0

>20.0

Broad

Catawba

Yadkin

Roanoke

Cape Fear

Neuse

Tar-Pamlico

Charlotte

Concord

Fayetteville

Raleigh

Durham

Asheboro

High Point

Winston-Salem Greensboro

(A)

! !

! !

! !

!

!

!

Change in FSR

-2.2 – -1.0 species

-0.9 – 0.0 species

0.1 – 1.0 species

1.1 – 3.1 species

Charlotte

Concord

Fayetteville

Raleigh

Durham

Asheboro

High Point

Winston-Salem Greensboro

(B)

! !

! !

! !

!

!

! Change in FSR

-3.8 – -1.0 species

-0.9 – 0.0 species

0.1 – 1.0 species

1.1 – 1.6 species

Charlotte

Concord

Fayetteville

Raleigh

Durham

Asheboro

High Point

Winston-Salem Greensboro

(C)

! !

! !

! !

!

!

!

Change in FSR

-3.8 – -1.0 species

-0.9 – 0.0 species

0.1 – 1.0 species

1.1 – 1.9 species

Charlotte

Concord

Fayetteville

Raleigh

Durham

Asheboro

High Point

Winston-Salem Greensboro

(D)

USDA Forest Service, Southern Research Station 59 e-General Technical Report SRS-246

CHAPTER 5

Figure 5.7—Predicted 1991–2010 baseline FSR values across the study area (A), and change in FSR under climate projections by 

2060 (B), 2060 impervious projections (C), and 20-percent withdrawals (D) using the four-variable BRT model.

and increasing FSR. In contrast to the climate scenario, 

the flow variables for the impervious cover scenario 

changed very little from the 1991–2010 average over the 

entire study area (table 5.6), which may help explain why 

there were no significant changes in average FSR across 

all HUC-12s for that scenario. Increases in impervious 

surfaces are predicted to occur in and around urban areas 

(USEPA 2007) and likely would not impact all HUC-12s 

within a region the way that climate change could. For 

example, even though increases in MaxCMS are positively 

correlated with FSR, when QFRAC_IMP is high, FSR 

tends to be low (fig. 5.6). Conversely, when QFRAC_

IMP is low, FSR tends to be high, especially in larger 

streams with higher MaxCMS. Some level of interaction 

is expected among flow attributes that summarize 

information across broad spatial scales; however, such 

findings are essential for supporting decision makers by 

giving them the tools and information needed to manage 

water resources when faced with multiple sources of 

change. For example, limiting water withdrawals in an 

undeveloped catchment to maintain or enhance FSR 

may not result in the desired endpoint if land use change 

results in increases in impervious cover and thus increases 

in QFRAC_IMP. Therefore, focusing on a management 

action that addresses only one streamflow component 

or that does not take into consideration non-stationarity 

principles (Milly and others 2008) could cause an overestimation of water availability and result in a significant 

over-allocation of the resource. 
!( !( !( !( !( !( !( !( !(

>1 species lost

Broad

Catawba Yadkin

Roanoke

Cape Fear

Neuse

Tar-Pamlico

Charlotte

Concord

Fayetteville

Raleigh

Durham

Asheboro

High Point

Winston-Salem Greensboro

(A)

! !

! !

! !

!

!

!

Charlotte

Concord

Fayetteville

Raleigh

Durham

Asheboro

High Point

Winston-Salem Greensboro

(B)

>1 species lost

! !

! !

! !

!

!

! Charlotte

Concord

Fayetteville

Raleigh

Durham

Asheboro

High Point

Winston-Salem Greensboro

(C)

>1 species lost

! !

! !

! !

!

!

! Charlotte

Concord

Fayetteville

Raleigh

Durham

Asheboro

High Point

Winston-Salem Greensboro

(D)

>1 species lost

60

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Figure 5.8—HUC-12s in the North Carolina Piedmont predicted to lose more than one fish species based on the 5-percent (A), 

10-percent (B), 15-percent (C), and 20-percent (D) withdrawal scenarios.

The value of a strong hydrologic foundation cannot be 

understated for supporting a broader understanding of 

the connection between changes in water availability and 

sustaining the long-term viability of fish assemblages. 

The modeling approach presented in this paper was vital 

for systematically assessing regional-scale effects and 

identifying areas of concern (i.e., “hot spots”) where the 

combined effects of land cover change, climate change, 

and/or streamflow alteration may threaten water resources. 

Once hot spots are identified, fine-scale, physically based 

models of higher temporal resolution could potentially 

be applied to those areas of concern to provide more 

quantitative estimates of changes in water availability 

and support sub-monthly ERFMs using more sitespecific inputs. 

Response of Fish Species Richness to 

Hydrologic Change 

Maintenance of hydrologic variability is critical to 

protecting biodiversity and maintaining the integrity 

of aquatic, riparian, and wetland ecosystems, and is 

the foundation of the Natural Flow Regime Paradigm 

(NFRP) presented by Poff and others (1997). Decades of 

observation of the effects of human alteration of natural 

flow regimes have established that streamflow variability 

is critical for maintaining the ecological integrity of river 

systems because many aquatic species have developed 

life-history strategies in response to these flow attributes 

(Hill and others 1991; Lytle and Poff 2004; Mims and 

Olden 2012, 2013; Poff and Ward 1989; Postel and Richter 

2012; Richter and others 1997; Stalnaker 1990). The 

coefficient of monthly streamflow variability (FloVar) was 
USDA Forest Service, Southern Research Station 61 e-General Technical Report SRS-246

CHAPTER 5

Table 5.5—Number (percentage, in parentheses) of 12-digit Hydrologic Unit Code (HUCs), by river basin, with a 

predicted decrease in fish species richness under each scenario

Climate 

change

Impervious 

cover Change in flow (water withdrawal)c

Basin

Number of 

HUCs in basin 2041–2060b 2041–2060b -5% -10% -15% -20% -25%

BRD 49 3 (6%) 15 (31%) 13 (27%) 13 (27%) 29 (59%) 49 (100%) 48 (98%)

CPF 215 87 (40%) 128 (60%) 148 (69%) 151 (70%) 177 (82%) 207 (96%) 211 (98%)

CTB 100 45 (45%) 50 (50%) 48 (48%) 66 (66%) 87 (87%) 98 (98%) 98 (98%)

NEU 134 39 (29%) 62 (46%) 50 (37%) 53 (40%) 54 (40%) 81 (60%) 109 (81%)

ROA 97 25 (26%) 36 (37%) 45 (46%) 31 (32%) 40 (41%) 61 (63%) 61 (63%)

TAR 85 33 (39%) 23 (27%) 57 (67%) 48 (56%) 43 (51%) 44 (52%) 59 (69%)

YAD 206 62 (30%) 95 (46%) 98 (48%) 123 (60%) 148 (72%) 189 (92%) 191 (93%)

All basinsa 886 294 (33%) 409 (46%) 459 (52%) 485 (55%) 578 (65%) 729 (82%) 777 (88%)

a

 Bolded values represent a significant difference (p <0.05) between current conditions and predicted values (note that significance was tested for the 

region as a whole, not per basin). 

b

 2041–2060 represents the time period for the climate change and impervious surface scenarios. 

c

 Percentage values from -5 to -25 represent water withdrawal scenarios. 

BRD = Broad; CPF = Cape Fear; CTB = Catawba; NEU = Neuse; ROA = Roanoke; TAR = Tar-Pamlico; YAD = Yadkin.

Table 5.6—Average differencea between scenarios and 2010 predictions (scenario - 2010) for 

changes in climate and imperviousness in major river basins across the North Carolina Piedmont 

Scenario Variable BRD CPF CTB NEU ROA TAR YAD All basins

Climate MaxCMS 3.32 6.51 7.82 5.98 15.28 5.18 7.88 7.55

QFRAC_IMP -0.01 0.00 -0.01 0.00 0.00 0.00 0.00 0.00

FloVar -0.07 0.00 -0.07 -0.03 -0.02 -0.09 -0.05 -0.04

Imperviousness MaxCMS 0.01 0.24 0.21 0.51 0.04 0.03 0.18 0.21

QFRAC_IMP 0.00 0.03 0.03 0.05 0.00 0.00 0.03 0.03

FloVar 0.00 -0.01 -0.01 -0.02 0.00 0.00 -0.01 -0.01

a

 Positive values represent a predicted increase under scenarios. 

BRD = Broad; CPF = Cape Fear; CTB = Catawba; NEU = Neuse; ROA = Roanoke; TAR = Tar-Pamlico; YAD = Yadkin.

MaxCMS = maximum monthly streamflow; QFRAC_IMP = the proportion of surface flow coming from impervious surfaces; 

FloVar = measure of the coefficient of monthly streamflow variability. 

one of the important predictors in our model. The PDP plot 

(fig. 5.5), before it flattens out, generally indicates a strong 

positive response between FSR and increasing streamflow 

variability which is in keeping with the principles of the 

NFRP. Therefore, the strength of the response for FloVar 

in the model underscores the importance of maintaining 

streamflow variability in support of a thriving fish 

assemblage (Bunn and Arthington 2002, Carlisle and 

others 2010, Poff and Zimmerman 2010). 

Although the goal of restoring streamflow to its “natural” 

condition may be unachievable in moderately to highly 

degraded urban systems with high human demand for 

water or in systems with numerous reservoirs designed for 

water-supply purposes, it still may be possible to offset 

future alterations in water availability resulting from 

climate or land use change by implementing proactive 

strategies that maintain variable passing flows or flowby standards that are consistent with NFRP principles. 

For example, the NCEFSAB, which was tasked with 

developing a scientifically defensible approach to 

establishing flows that protect the ecological integrity of 

streams and rivers in North Carolina as required under 

Session Law 2010-143, suggested an 80–90-percent flowby (i.e., 80–90 percent of ambient modeled flow remains 

in the stream; NCEFSAB 2013) in combination with a 
62

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

critical low-flow component. Results of our plausible 

withdrawal scenarios are highly consistent with the 

NCEFSAB’s recommendations. 

Modeling Limitations 

Ecologically relevant flow metrics predicted by the WaSSI 

model were subject to similar uncertainties associated 

with other hydrologic models (Caldwell and others 2015), 

including uncertainty in climate, land cover, soil, and LAI 

input data, as well as uncertainty in the representation of 

the physical processes that govern streamflow magnitude 

and timing. Unlike calibrated models, the WaSSI model 

will be less sensitive to errors associated with expanding 

the model domain to catchments not included in the model 

calibration process and using the model to assess the effect 

of climate or land cover scenarios outside of the conditions 

for which it was calibrated. The overall accuracy of 

the model was considered satisfactory given the many 

uncertainties in model inputs, model representation of 

the physical system, and observed streamflow data (see 

Caldwell and others 2015). We acknowledge that there is 

considerable uncertainty in the prediction of future climate 

and land cover; however, the projections we used provided 

a reasonable scenario of how they may change, and this 

was supported by our model validation results (table 5.1). 

We were not able to capture some of the more specific 

sub-monthly streamflow attributes that may be important 

for fish migration and reproduction (e.g., annual daily 

minimum and maximum streamflow, daily streamflow 

exceedances and recession rates; see Kennen and others 

2007, Konrad and others 2008, Olden and Poff 2003) 

because the WaSSI model functions at a monthly time 

step. However, even with this limitation, we were able to 

develop a significant four-variable BRT model that had 

good predictive power and helped to better understand the 

potential effects of increasing water withdrawal on FSR in 

the North Carolina Piedmont region. Hydrologic models 

vary in their levels of complexity, temporal and spatial 

resolution, and required level of calibration. Detailed 

and highly parameterized fine-resolution models such as 

distributed physically based watershed and rainfall-runoff 

models are well suited for smaller domains but can be 

computationally expensive and difficult to parameterize at 

larger scales. In contrast, easily parameterized regionalscale models such as monthly water balance models 

(e.g., WaSSI, the USGS Monthly Water Balance Model; 

Hay and McCabe 2002) are useful for assessing broad 

implications of streamflow alteration at a large scale and 

identifying potential water-limited areas but may have 

difficulty resolving unique sub-watershed-scale physical 

and ecological processes and associated anthropogenic 

effects. Leveraging the benefits of both large-scale models 

with high-resolution models has the potential to allow 

more robust evaluations of the effects of water withdrawal 

on aquatic ecosystems. WaSSI, as demonstrated in this 

paper, can be used in conjunction with biological data 

to develop flow-ecology models that assess broad-scale 

regional impacts and identify specific catchments of 

concern (“hot spots”) where the combined effects of land 

cover change, climate change, and/or flow alteration may 

threaten water resources. 

There are also some limitations implicit in flow-ecology 

models constructed using machine learning techniques 

such as the BRT model presented in this paper. The 

strength of BRT models is that they improve on the 

basic averaging algorithm used in random forest (De’ath 

2007); however, the improvements in prediction accuracy 

may come at the expense of some loss of interpretation. 

For example, many of the advanced machine learning 

techniques, such as BRT, have a tendency to overfit the 

data (Aertsen and others 2010). Goodness of fit measures 

and k-fold cross-validation techniques, as applied in 

this study, have been implemented to help practitioners 

understand and offset this limitation (Elith and others 

2008). However, care should be taken to make sure 

results are not influenced by spatial sorting bias or spatial 

autocorrelation (Hijmans 2012, Randin and others 2006). 

A general weakness of BRT models is that they are not as 

familiar to scientists and managers as modeling methods 

such as multiple linear regression. Thus, explaining how 

BRT models work and how to interpret the results in a 

manner that supports management decisions can be a 

challenge. The general robustness and greater predictive 

power of machine learning techniques greatly outweigh 

their limitations and, as their application becomes more 

commonplace in ecology, especially for modeling nonlinear relationships, their level of acceptance in the 

management arena will also increase.

Improvements/Future Work

The use of FSR as the primary measure of fish assemblage 

integrity as part of this study provides a level of simplicity 

and parsimony that supports scientific reproducibility and 

management application at the State and regional level. 

However, richness is only one measure of assemblage 

integrity, and alone it may limit broader interpretation of 

the hydrologic effects on fish reproduction, life-history 

processes, and species of special concern. Moreover, there 

is a need to better understand underlying mechanisms 

(Poff 2018) that explain local abundance and regional 

distributions of fish species. Examining fish species 

traits is one such method that has been shown to be a 

powerful tool in ecology for identifying trends within 

and among species assemblages (Statzner and others 
USDA Forest Service, Southern Research Station 63 e-General Technical Report SRS-246

CHAPTER 5

2001) and for representing measurable characteristics 

based on morphological, physiological, or life-history 

attributes (Violle and others 2007). Additionally, there is 

a need to more broadly implement trait-based research 

in fish ecology, which to date has largely been focused 

on terrestrial plants and aquatic invertebrates (Verberk 

and others 2013). Therefore, results of this study could 

be enhanced through the applications of functional 

traits as a means to better understand the effects of 

hydrologic alteration on fish assemblages and support the 

conservation of fish species of special concern in the North 

Carolina Piedmont. 

CONCLUSIONS

In this study, streamflow indices including the maximum 

monthly streamflow and the coefficient of streamflow 

variability were shown, in part, to be particularly important 

for supporting the richness of fish assemblages in the 

North Carolina Piedmont. The results strongly support 

other studies that have shown that as the magnitude of 

high flows and natural variability in annual streamflow 

is altered, the richness of species with life-history and 

behavioral constraints that rely on annual high-flow 

patterns or fluctuations in flow for reproduction may be 

reduced. Implementing water management measures 

that meet the constraints of the NFRP has been a major 

challenge for management agencies. Developing practical 

flow-protection standards that limit groundwater and 

surface water withdrawals and interbasin transfers, or 

the implementation of designed flow releases that protect 

essential streamflow variability, have been difficult 

to achieve or have been met with strong resistance or 

legal actions. Therefore, it is essential that management 

strategies developed in collaboration with stakeholders 

that minimize flow alteration strive to conserve FSR. 

Improved water management incentives need to be 

established within the constraints of existing water law 

and government statutes that support designated uses, 

meet existing regulatory requirements, and promote a 

balance between water supply to support human needs and 

conservation of biological integrity.
64

CHAPTER 6 

Synthesis

SUMMARY

Hydrologic models are commonly used to develop 

streamflow hydrographs for ecological flow studies 

because they can simulate streamflow under baseline 

conditions and an infinite number of scenarios of 

streamflow alteration. Hydrologic models vary in their 

levels of complexity. For example, detailed and highly 

parameterized fine-resolution models are well suited for 

smaller domains but can be computationally expensive and 

difficult to parameterize at larger scales. In contrast, easily 

parameterized large-scale models are useful for assessing 

broad implications of streamflow alteration at a greater 

spatial scale and identifying potentially water-limited areas 

(i.e., “hot spots”) but may have difficulty resolving unique 

sub-watershed-scale physical processes and associated 

anthropogenic effects altering streamflow. The primary 

objective of this study was to provide resource managers 

and environmental flow practitioners with some insight 

into the relative error in streamflow predictions among 

a subset of hydrologic models commonly used for water 

supply assessment, environmental flow studies, and 

climate change predictions in the Southeastern United 

States (SEUS) and Puerto Rico (PR). This effort was 

designed to evaluate, quantify, and compare the magnitude 

and investigate the potential causes of error associated 

with predicted streamflows from seven hydrologic models 

of varying complexity and calibration strategy. In addition, 

we postulated that leveraging the benefits of both largescale models and high-resolution models will allow more 

robust evaluations of the effects of changes in water 

availability on aquatic ecosystems. Such an approach 

provides water managers with information necessary to 

better balance water resources needed to support aquatic 

assemblages while conserving water for long-term human 

uses across broad regions. 

We began by creating an inventory of existing hydrologic 

modeling efforts in the SEUS and PR. We contacted 95 

individuals from 64 unique organizations throughout 

the region, and 20 agreed to be interviewed. Details on 

their modeling efforts including the model developer, 

intended purpose, model framework, spatial extent, spatial 

and temporal resolution, time period simulated, model 

inputs, model outputs, and elements of environmental 

change represented. Validation procedure, criteria, and 

results were collected. Of the 64 organizations solicited, 

19 represented Federal agencies, 11 represented State 

agencies, 32 represented universities, and 2 represented 

private sector organizations. The 20 individuals 

interviewed were developing and using hydrologic models 

across the SEUS and PR to answer broad questions 

regarding the impacts of environmental change on water 

resources. With the rapid pace of computing technology 

and growth of modeling approaches, as well as changing 

threats to watersheds across the landscape, it is expected 

that this inventory of hydrologic modeling efforts will 

continue to evolve, and therefore the findings presented 

in this report represent a snapshot of approaches and 

knowledge gaps in hydrologic modeling for flow-ecology 

science and environmental change impacts.

We then quantified and compared the magnitude and 

investigated the potential causes of error associated with 

predicting streamflows for 195 U.S. Geological Survey 

(USGS) continuous record gauging stations using seven 

hydrologic models of varying complexity and calibration 

strategy. Models included the Hydrological Simulation 

Program-Fortran (HSPF) model; the Monthly Water 

Balance Model (MWBM); two parameterizations of the 

Precipitation-Runoff Modeling System (PRMS) model; 

three parameterizations of the WATER model, based 

on TOPMODEL (a physically based, semi-distributed 

topographical watershed model); the Soil and Water 

Assessment Tool (SWAT) model; the Generalized 

Watershed Loading Function (GWLF)-based WaterFALL®

model; and the Water Supply Stress Index (WaSSI) model. 

After model simulations were performed and statistical 

measures of model performance were computed, a 

model comparison workshop was convened in which 14 

model developers and users representing eight different 

governmental, academic, and consultant organizations 

came together to discuss the advantages and disadvantages 

of different modeling approaches, sources of input data, 

and model calibration techniques. Findings from this study 

indicated that no specific hydrologic model is superior 

to the others evaluated for all sites and for all measures 

of model performance. Differences among model 

predictions were as likely to be related to differences 

in model calibration strategy as they were related to 

differences in model structure as increasing calibration 

intensity generally improved model fit. In addition, results 

indicated that some large-scale flow routing models 
USDA Forest Service, Southern Research Station 65 e-General Technical Report SRS-246

CHAPTER 6

(e.g., WaSSI, MWBM) have comparable performance 

to the more complex, fine-scale models at a monthly 

time step, and large-scale models and fine-scale models 

have similar sensitivity to changes in precipitation and 

air temperature at the annual scale while sensitivity to 

changes in precipitation and temperature are different at 

the monthly scale. We do not provide recommendations 

for the use of any particular model that was included in 

this study. Instead, we stress that it is incumbent upon 

resource managers, environmental flow practitioners, and 

policymakers to consider the expertise of the modeler, the 

applicability of a model to a particular resource problem, 

the context to which the model is being applied, the 

timeframe being evaluated, the scale of interest, and the 

important components of the flow regime that may be used 

for model calibration to minimize error across the targeted 

range of flows and thus improve flow-ecology relations.

Lastly, we conducted a demonstration study for regionalscale flow-ecology response modeling for environmental 

change impact assessment on fish species richness (FSR) 

in the North Carolina Piedmont by evaluating plausible 

scenarios of changes in water withdrawals, climate, and 

impervious surfaces. Fish species richness and streamflow 

predictions were then used to build a boosted regression 

tree (BRT) flow-ecology model which was used to predict 

the relationship between FSR and a subset of ecologically 

relevant flow metrics (ERFMs) in all 12-digit Hydrological 

Unit Code catchments (HUC-12s) in the North Carolina 

Piedmont. The results of the model were then used to 

predict FSR under three plausible scenarios of future water 

withdrawals, climate change, and increases in impervious 

surfaces (see fig. 5.7). Finally, a “hot spot” analysis was 

used to identify individual HUC-12s that were most likely 

to be affected by changes in water availability. Using 

this approach, the BRT flow-ecology model was able to 

explain 50 percent of the variability in observed FSR 

using river basin, mean annual maximum monthly flow, 

flow variability, and the proportion of flow originating on 

impervious surfaces as explanatory variables. Fish species 

richness decreased with increasing withdrawals, and a 

reduction in flow of 25 percent was predicted to result in 

a significant average loss of one species across the region. 

Similarly, FSR was predicted to decrease with projected 

increases in impervious cover through mid-century but 

only in areas proximal to growing urban centers such as 

Charlotte, Greensboro, Raleigh-Durham, and Fayetteville. 

Projected climate change by mid-century increased FSR 

by 0.35 species on average across the region but ranged 

from a decrease of 2.19 species in some watersheds to 

an increase of 3.10 species in others. This pilot project 

demonstrated that the use of a large-scale hydrologic 

model coupled with flow-ecology models can be useful for 

systematically assessing regional-scale changes in water 

availability and identifying areas of concern (i.e., “hot 

spots”) where the combined effects of land cover change, 

climate change, and/or streamflow alteration may threaten 

water resources. The study also indicates that fine-scale, 

physically based models of higher temporal resolution 

could potentially be applied to those areas of concern to 

provide higher resolution quantitative estimates of changes 

in water availability and support sub-monthly ERFMs 

using more site-specific inputs. 

LIMITATIONS

While this report likely represents what may be the most 

rigorous evaluation of the use of hydrologic models for 

flow-ecology science compiled for the SEUS and PR, 

we acknowledge that it only provides an overview of the 

many tools and techniques available to environmental 

flow practitioners and water resource managers. Below we 

highlight some of the potential limitations of this study.

1. Only a subset of the available hydrologic models were 

considered in this study.

2. The models evaluated were not developed using 

the same calibration objective functions and input 

datasets, making it difficult to separate differences 

in model performance that may be related to the 

model framework (e.g., HSPF or PRMS) from 

differences associated with the choice of model inputs 

and calibration.

3. New methods are being developed all the time. The 

findings in this report are not stationary and should be 

reevaluated from time to time.

4. The capacity to fully understand non-stationary 

conditions associated with climate change requires 

rigorous calibration of models and careful attention to 

model inputs and representation of physical processes 

that may assume stationarity. For example, adjusting 

precipitation and other climate variables may improve 

model fit for historical flow observations, but these 

adjustments may not be appropriate when using the 

model to make projections using other sources of 

climate input (e.g., future climate change scenarios). 

Incorporating dynamic parameterization schemes into 

existing hydrologic models may provide a pathway 

forward for dealing with non-stationarity.

5. Models evaluated were applied to several basins in 

the SEUS region. The relative model performance 

may be different in other hydroclimatic settings 

(e.g., snowmelt-dominated streams and streams in 

arid climates). 
66

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

6. There are tradeoffs between regional- and fine-scale 

models. Regional-scale models are often easier to 

parameterize but are often of a coarser resolution 

in space and time. Fine-scale models are often 

more difficult to parameterize but can have finer 

resolution and smaller time steps. Resource managers 

should consider the desired resolution of streamflow 

predictions when selecting a model for a particular 

resource problem and balance the need for that 

resolution with the expense in input data requirements, 

computational limitations, and the acceptance of a 

desired level of uncertainty. 

7. We demonstrated that similar levels of model 

performance may be obtained at the monthly time step 

using regional- and fine-scale hydrologic models. We 

have also shown that regional-scale and some fine-scale 

hydrologic models predict similar changes in runoff for 

a given change in climate inputs. However, these results 

only provide a demonstration of the potential of multiscale modeling approaches to evaluate environmental 

change effects on streamflow and ecological response. 

Due to differences in land cover input data, we were 

not able to determine whether there would be similar 

streamflow response to land cover change among 

the models evaluated. Additional study is required to 

determine the best way to use regional- and fine-scale 

models to identify hot spots or change and to develop 

flow-ecology relations, respectively. Models should 

be developed using the same inputs and calibration 

objective functions and then used to evaluate the same 

scenarios of climate and land cover change. 

8. There are tradeoffs between calibrating models to best 

match observed high-flow or low-flow portions of the 

hydrograph that may affect flow-ecology modeling. 

Model calibration is generally intended to capture the 

variability and the central tendency of streamflow. It is 

nearly impossible to calibrate models to fit the entire 

range of observed streamflows because adjusting model 

parameters to fit a portion of the flow regime has an 

effect on how well the model fits observed streamflows 

outside of that range. 

9. Many of the advanced machine learning techniques 

used in flow-ecology modeling, such as BRT, have a 

tendency to overfit the data. Goodness of fit measures 

and cross-validation techniques, as applied in the pilot 

study, have been implemented to help practitioners 

understand and offset this limitation. However, care 

should be taken to make sure results are not influenced 

by spatial sorting bias or spatial autocorrelation. 

10. A general weakness of BRT models is that they 

are not as familiar to scientists and managers as 

modeling methods such as multiple linear regression. 

Thus, explaining how BRT models work and how 

to interpret the results in a manner that supports 

management decisions can be a challenge.

11. This study focused primarily on water quantity. While 

understanding the effects of water withdrawals on 

streams and other freshwater bodies in support of 

water availability and human consumptive use is 

germane to this report, the effects of changes in water 

quality should not be overlooked. Anthropogenic 

factors can be cumulative, and it is not just the 

quantity but also the quality of water available to 

ecosystems that may exacerbate conditions and affect 

the long-term health and vitality of aquatic systems.

12. The monthly time step of the WaSSI model may have 

limited our interpretation of FSR response to flow 

changes because we were not able to include some of 

the more specific sub-monthly streamflow attributes 

that may be important to fish migration, survival, and 

reproduction (e.g., August low flows or spring high 

flows). Further studies using daily flow attributes in 

combination with fish species traits (e.g., life history, 

habitat preference, and reproductive strategy) or other 

metrics that better encapsulate mechanistic response 

to changes in flow may improve our understanding of 

fish-flow relationships.

Despite these limitations, it is our hope that this report will 

provide environmental flow practitioners, water resource 

managers, and stakeholders in the SEUS with an informed 

pathway for developing the capacity to link streamflow 

and ecological response and understand some of the 

limitations associated with hydrologic and flow-ecology 

modeling efforts. 

FUTURE DIRECTIONS

This study highlighted several areas that require additional 

research regarding the prediction of streamflow for 

flow-ecology science in the SEUS and PR. Our model 

inventory indicated that while some areas in the region 

were well studied (e.g., the Apalachicola-ChattahoocheeFlint Basin in Georgia, Alabama, and Florida), other 

areas were not (e.g., Gulf Coast States and PR). Future 

work should strive to build modeling capacity in these 

under-represented regions (see also table 2.3). In 

addition to incomplete spatial extent across the region, 

our study indicated that hydrologic models in general 

have difficulty in predicting extremely high and low 
USDA Forest Service, Southern Research Station 67 e-General Technical Report SRS-246

CHAPTER 6

flows. These findings may have implications for the 

development of flow-ecology response models because 

it is often the low flows (baseflows), annual-flow pulses, 

and seasonality of high flows that provide the conditions 

necessary to support natural-assemblage complexity 

(Matthews 2005, Poff and Ward 1989, Poff and others 

1997, Richter and others 1997, Stanford and others 1996). 

While improved calibration of model parameters could 

partially overcome limitations of modeling assumptions 

and simplifications of hydrologic processes, there is a 

clear need to understand the sources of uncertainty in 

prediction at the low and high end of the flow regime (e.g., 

potentially unrepresentative model input data including 

precipitation, land cover, and soil properties vs. techniques 

used to model specific hydrologic processes). Watersheds 

simulated in this model comparison study had minimal 

hydrologic alteration due to withdrawals, interbasin 

transfers, or dams so that we could assess the performance 

of participating models in predicting the natural flow 

regime. However, unaltered watersheds in the SEUS may 

be the exception rather than the norm. Unfortunately, 

comprehensive georeferenced regional databases that 

describe the locations and management protocols of dams, 

locations and volumes of withdrawals, locations and 

volumes of wastewater outfalls and interbasin transfers, 

and stormwater flow routing are incomplete or lacking, 

but the ongoing congressionally mandated USGS National 

Water Census (https://water.usgs.gov/watercensus/) is 

helping to meet this need by providing broader access 

to hydrologic and water budget data across the United 

States. Future research should continue to assemble these 

data using a consistent methodology so that the effects 

of flow alteration across regions can be assessed. In 

addition to flow alterations by dams and surface water 

withdrawals, the effects of groundwater pumping on 

streamflow at the regional scale is not well understood. 

More than 50 percent of the population in the SEUS 

relies on groundwater wells for drinking water (USEPA 

2012), and 76 percent of all agricultural withdrawals in 

the SEUS originate from groundwater (Maupin and others 

2014), but these groundwater withdrawals are largely 

unregulated and undocumented across the region. In total, 

groundwater withdrawals account for 25 percent of all 

withdrawals in the SEUS and as much as 69 percent of all 

withdrawals at the State level (Maupin and others 2014). 

Additional study will be required to better understand 

surface and groundwater interactions and the effects of 

groundwater pumping on surface water flow regimes, 

particularly in the Atlantic Coastal Plain where surface and 

shallow groundwater are more tightly coupled than other 

ecoregions of the SEUS. 

The development of hydrographic information at 

ungauged locations and the aggregation of fish assemblage 

data for the Piedmont region of North Carolina provide 

two important components of the Ecological Limits 

of Hydrologic Alteration framework (Poff and others 

2010) that can be used to support future flow-ecology 

modeling efforts that address non-stationarity principles 

(Milly and others 2008) and incorporate more temporal 

and quantitative elements of ecological assemblages, 

such as species rates. Rather than a static measure that 

represents a snapshot of a condition (i.e., a state) derived 

from a single measurement in time (such as FSR used 

in this study), ecological responses in a rates approach 

reflect temporal change (i.e., a rate) and are thus reliant 

on repeated measurements made over time (Wheeler and 

others 2018). The concept of non-stationarity presents a 

management challenge because the definition of natural, 

baseline, or reference flow conditions is dependent not 

only on human and natural influences on the landscape 

but also on our continually changing perspective of what 

a baseline condition represents. Hydrologic baselines are 

shifting and therefore reliance on restoration to a reference 

condition for either hydrologic or ecological systems is 

unsustainable. To meet this challenge, it will be important 

to implement ecological flows using flexible and adaptive 

management approaches that will support the long-term 

resiliency of valued ecosystems (Poff 2018). While the use 

of FSR as a measure of fish assemblage integrity provides 

a level of simplicity and parsimony that supports scientific 

reproducibility and management application at the State 

and regional level, it may limit broader interpretation of 

the hydrologic effects on fish reproduction, life-history 

processes, and species of special concern. Moreover, there 

is a need to better understand underlying mechanisms 

(Poff 2018) that explain local abundance and regional 

distributions of fish species. Examining fish species traits 

is one such method that has been shown to be a powerful 

tool in ecology for identifying trends within and among 

species assemblages and for characterizing morphological, 

physiological, or life-history attributes. We envision 

that the results of this study could be enhanced through 

the application of a rates approach and the inclusion of 

functional traits as a means to gain a better understanding 

of the underlying mechanistic relationship between 

hydrologic alteration and fish assemblage response. Such 

an understanding would better support the conservation 

of fish species of special concern in the North Carolina 

Piedmont and is essential to water resource management.
68

Acknowledgments

We are indebted to the many individuals and 

organizations that willingly participated in this 

modeling study and provided the data and flow 

model output that were integral to the hydrologic model 

comparisons and the development of flow-ecology 

relations. We thank Jerry McMahon, Director of the U.S. 

Department of the Interior Southeast Climate Adaptation 

Science Center, for his guidance and support throughout 

the duration of this project. We thank Timothy Shortley, 

North Carolina State University, Department of Forestry 

and Environmental Resources, Center for Geospatial 

Analytics, for his contribution to the modeling inventory 

(chap. 2). We are indebted to Bryn Tracy of the North 

Carolina Division of Water Resources who collected and 

curated the fish species dataset which was integral to 

this project and many others. We thank North Carolina 

State University for providing the facilities for a model 

comparison workshop among organizations and modelers 

participating in this study. Jeremy Wyss (Tetra Tech) and 

Pushpa Tuppad (Texas A&M University) were responsible 

for calibrating the basin-scale Hydrological Simulation 

Program-Fortran (HSPF) and Soil and Water Assessment 

Tool (SWAT) models, respectively. The HSPF and 

SWAT model applications were supported by the U.S. 

Environmental Protection Agency Office of Research 

and Development’s National Center for Environmental 

Assessment under the guidance of Dr. Thomas Johnson; 

however, the views expressed in this paper represent those 

of the authors and do not necessarily reflect the views or 

policies of the U.S. Environmental Protection Agency. 

Funding for the Water Supply Stress Index (WaSSI) 

model analysis was provided by the U.S. Department of 

Agriculture (USDA) Forest Service, Southern Research 

Station, Eastern Forest Environmental Threat Assessment 

Center and Coweeta Hydrologic Laboratory. Funding 

for this study and for a model comparison workshop was 

provided by the Southeast Climate Adaptation Science 

Center and the U.S. Geological Survey’s (USGS) Water 

Availability and Use Science Program. This report has 

benefitted from peer review from Kai Duan (USDA Forest 

Service Southern Research Station, Center for Forest 

Watershed Research, Coweeta Hydrologic Laboratory) and 

Pam Reilly (USGS New Jersey Water Science Center).
USDA Forest Service, Southern Research Station 69 e-General Technical Report SRS-246

References

Acreman, M.; Dunbar, M.; Hannaford, J. [and others]. 2008. Developing 

environmental standards for abstractions from UK rivers to implement 

the EU Water Framework Directive. Hydrological Sciences Journal. 

53: 1105–1120. DOI:10.1623/hysj.53.6.1105.

Aertsen, W.; Kint, V.; Van Orshoven, J. [and others]. 2010. Comparison 

and ranking of different modelling techniques for prediction of site 

index in Mediterranean mountain forests. Ecological Modelling. 

221: 1119–1130. DOI:10.1016/j.ecolmodel.2010.01.007. 

Alley, W.M.; Evenson, E.J.; Barber, N.L. [and others]. 2013. Progress 

toward establishing a national assessment of water availability and 

use. U.S. Geological Survey Circular 1384. 34 p. http://pubs.usgs.gov/

circ/1384. [Date last accessed: July 3, 2019].

Annear, T.; Chisholm, I.; Beecher, H. [and others]. 2004. Instream flows 

for riverine resource stewardship, revised edition. Cheyenne, WY: 

Instream Flow Council. 268 p.

Archfield, S.A.; Kennen, J.G.; Carlisle, D.M. [and others]. 2013. An 

objective and parsimonious approach for classifying natural flow 

regimes at a continental scale. River Research and Applications. 

30: 1166–1183. DOI:10.1002/rra.2710.

Arthington, A.H.; Bernardo, J.M.; Ilhéu, M. 2014. Temporary rivers: 

linking ecohydrology, ecological quality and reconciliation ecology. 

River Research and Applications. 30: 1209–1215. DOI:10.1002/

rra.2831.

Arthington, A.H.; Bunn, S.E.; Poff, N.L.; Naiman, R.J. 2006. The 

challenge of providing environmental flow rules to sustain 

river ecosystems. Ecological Applications. 16: 1311–1318. 

DOI:10.1890/1051-0761(2006)016[1311:TCOPEF]2.0.CO;2.

Arthington, A.H.; Kennen, J.G.; Stein, E.D.; Webb, J.A. 2018. 

Recent advances in environmental flows science and water 

management—innovation in the Anthropocene. Freshwater Biology. 

63(8): 1022– 1034.

Averyt, K.; Fisher, J.; Huber-Lee, A. [and others]. 2011. Freshwater 

use by US power plants: electricity’s thirst for a precious resource. 

Cambridge, MA: Union of Concerned Scientists. 52 p. 

Averyt, K.; Meldrum, J.; Caldwell, P. [and others]. 2013. Sectoral 

vulnerabilities to changing water resources in the coterminous U.S. 

Environmental Research Letters. 8: 035026. DOI:10.1088/1748-

9326/8/3/035046.

Beaujouan, V.; Durand, P.; Ruiz, L. 2001. Modelling the effect of the 

spatial distribution of agricultural practices on nitrogen fluxes in rural 

catchments. Ecological Modelling. 137(1): 93–105. DOI: 10.1016/

S0304-3800(00)00435-X.

Beven, K.J.; Kirkby, M.J. 1979. A physically based, variable contributing 

area model of basin hydrology. Hydrological Sciences Bulletin. 

24: 43–69.

Bicknell, B.R.; Imhoff, J.C.; Kittle, J.L. 2001. Hydrological Simulation 

Program-Fortran (HSPF). User’s manual for release 12. Athens, GA: 

U.S. Environmental Protection Agency, National Exposure Research 

Laboratory. 845 p. 

Bicknell, B.R.; Imhoff, J.C.; Kittle, J.L. 2005. HSPF version 12.2 user’s 

manual. Athens, GA: U.S. Environmental Protection Agency, National 

Exposure Research Laboratory. [Number of pages unknown].

Biemans, H.; Haddeland, I.; Kabat, P. [and others]. 2011. Impact of 

reservoirs on river discharge and irrigation water supply during 

the 20th century. Water Resources Research. 47: W03509. 

DOI:10.1029/2009WR008929.

Bierwagen, B.G.; Theobald, D.M.; Pyke, C.R. [and others]. 2010. 

National housing and impervious surface scenarios for integrated 

climate impact assessments. Proceedings of the National Academy 

of Sciences of the United States of America. 107: 20887–20892. 

DOI:10.1073/pnas.1002096107. 

Boyer, E.W.; Hornberger, G.M.; Bencala, K.E.; McKnight, D.M. 1996. 

Overview of a simple model describing variation of dissolved organic 

carbon in an upland catchment. Ecological Modeling. 85: 183–188. 

DOI:10.1016/0304-3800(95)00049-6.

Breiman, L.; Friedman, J.H.; Olshen, R.A.; Stone, C.J. 1984. 

Classification and regression trees. New York, NY: Chapman & Hall. 

358 p. 

Brown, L.R.; May, J.T.; Rehn, A.C. [and others]. 2012. Predicting 

biological condition in southern California streams. Landscape and 

Urban Planning. 108: 17–27. DOI:10.1016/j.landurbplan.2012.07.009. 

Bunn, S.E.; Arthington, A.H. 2002. Basic principles and ecological 

consequences of altered flow regimes for aquatic biodiversity. 

Environmental Management. 30: 492–507. DOI:10.1007/s00267-002-

2737-0.

Burnash, R.J.C. 1995. The NWS river forecast system - catchment 

modeling. In: Singh, V.P., ed. Computer models of watershed 

hydrology. Littleton, CO: Water Resources Publications: 311– 366.

Burnash, R.J.C.; Ferral, R.L.; McGuire, R.A. 1973. A generalized 

streamflow simulation system: conceptual modeling for digital 

computers. Technical Report, Joint Federal and State River Forecast 

Center. Sacramento, CA: U.S. National Weather Service and 

California Department of Water Resources. 204 p.

Caldwell P.V.; Kennen J.G.; Sun G. [and others]. 2015. A comparison 

of hydrologic models for ecological flows and water availability. 

Ecohydrology. 8: 1525–1546. DOI:10.1002/eco.1602.

Caldwell, P.V.; Sun, G.; McNulty, S.G. [and others]. 2012. Impacts of 

impervious cover, water withdrawals, and climate change on river 

flows in the Conterminous US. Hydrology and Earth System Sciences. 

16: 2839–2857. DOI:10.5194/hess-16-2839-2012.

Carlisle, D.M.; Wolock, D.M.; Meador, M.R. 2010. Alteration of 

streamflow magnitudes, and potential ecological consequences: a 

multiregional assessment. Frontiers in Ecology and the Environment. 

9: 264–270. DOI:10.1890/100053.

Chessman, B.C.; Jones, H.A.; Searle, N.K. [and others]. 2010. Assessing 

effects of flow alteration on macroinvertebrate assemblages in 

Australian dryland rivers. Freshwater Biology. 55: 1780–1800. 

DOI:10.1111/j.1365-2427.2010.02403.x. 
70

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Clapcott, J.; Young, R.; Goodwin, E. [and others]. 2011. Relationships 

between multiple land-use pressures and individual and combined 

indicators of stream ecological integrity. DOC Research and 

Development Series 326. Wellington, New Zealand: Department of 

Conservation. 57 p. 

Cohen, E.; Sun, G.; Zhang, L. [and others]. 2017. Quantifying the role of 

forested lands in providing surface drinking water supply for Puerto 

Rico. Gen. Tech. Rep. SRS-197-Addendum. Asheville, NC: U.S. 

Department of Agriculture Forest Service, Southern Research Station. 

20 p.

Conroy, M.J.; Allen, C.R.; Peterson, J.T. [and others]. 2003. Landscape 

change in the southern Piedmont: challenges, solutions, and 

uncertainty across scales. Conservation Ecology. 8: 3. DOI:10.5751/

ES-00598-080203. 

Crawford, N.H.; Linsley, R.K. 1966. Digital simulation in hydrology: 

Stanford watershed model IV. Technical Report No. 39. Stanford, CA: 

Stanford University Department of Civil Engineering. 210 p.

Dalton, M.S.; Jones, S.A., comps. 2010. Southeast Regional Assessment 

Project for the National Climate Change and Wildlife Science Center. 

Open-File Report 2010–1213. Reston, VA: U.S. Department of the 

Interior, U.S. Geological Survey: 38 p.

Davis, M. 2011. South Atlantic Conservation Cooperative SARP Aquatic 

Resource Management Project assessment of hydrologic model 

availability. http://www.southatlanticlcc.org/page/instream-flow. [Date 

accessed: August 23, 2017].

De’ath, G.; Fabricius, K.E. 2000. Classification and regression trees: a 

powerful yet simple technique for ecological data analysis. Ecology. 

81: 3178–3192. DOI:10.1890/0012-9658(2000)081[3178:CARTAP]2

.0.CO;2.

De’ath, G. 2007. Boosted trees for ecological modeling and prediction. 

Ecology. 88: 243–251. DOI:10.1890/0012-9658(2007) 88 

[243:BTFEMA]2.0.CO;2. 

Di Luzio, M.; Johnson, G.L.; Daly, C. [and others]. 2008. Constructing 

retrospective gridded daily precipitation and temperature datasets for 

the conterminous United States. Journal of Applied Meteorology and 

Climatology. 47: 475–497. DOI:10.1175/2007JAMC1356.1.

Doherty, J. 2008. FORTRAN 90 modules for implementation of 

parallelised, model-independent, model-based processing. Watermark 

Numerical Computing. 236 p.

Donigian, A.S., Jr. 2000. HSPF training workshop handbook and CD. 

Lecture #19: calibration and verification issues. U.S. Environmental 

Protection Agency Headquarters; Washington Information Center; 

10−14 January, 2000. Prepared for U.S. Environmental Protection 

Agency, Office of Water, Office of Science and Technology, 

Washington, D.C. Slide# L19-22. 

Duan, Q.Y.; Sorooshian, S.; Gupta, V.K. 1994. Optimal use of the 

SCE-UA Global Optimization Method for calibrating watershed 

models. Journal of Hydrology. 158: 265–284. DOI: 10.1016/0022-

1694(94)90057-4.

Dudgeon, D.; Arthington, A.H.; Gessner, M.O. [and others]. 2006. 

Freshwater biodiversity: importance, threats, status and conservation 

challenges. Biological Review. 81: 163–182. DOI:10.1017/

S1464793105006950.

Elith, J.; Leathwick, J.R.; Hastie, T. 2008. A working guide to 

boosted regression trees. Journal of Animal Ecology. 77: 802–813. 

DOI:10.1111/j.1365-2656.2008.01390.x. 

Emanuel, R.E.; Buckley, J.J.; Caldwell, P.V. [and others]. 2015. Influence 

of basin characteristics on the effectiveness and downstream reach 

of interbasin water transfers: displacing a problem. Environmental 

Research Letters. 10 (2015): 124005. DOI:10.1088/1748-

9326/10/12/124005.

Engel, V.; Stieglitz, M.; Griffin, K.; Williams, M. 2002. Canopy hydraulic 

conductance and catchment water balance: observations and modeling. 

Ecological Modelling. 154(3): 263– 288.

Falcone, J.A. 2011. Geospatial attributes of gages for evaluating 

streamflow. Digital spatial data set. http://water.usgs.gov/GIS/

metadata/usgswrd/XML/gagesII_Sept2011.xml. [Date accessed: 

August 23, 2017].

Falcone, J.A.; Carlisle, D.M.; Wolock, D.M.; Meador, M.R. 2010. 

GAGES: a stream gage database for evaluating natural and altered 

flow conditions in the conterminous United States. Ecology. 

91(2): 621.

Farmer, W.H.; Archfield, S.A.; Over, T.M. [and others]. 2014. A 

comparison of methods to predict historical daily streamflow 

time series in the Southeastern United States: U.S. Geological 

Survey Scientific Investigations Report 2014-5231. Reston, VA: 

U.S. Department of the Interior, U.S. Geological Survey. 34 p. 

DOI:10.3133/sir20145231.

Fisher, W.L.; Taylor, J.; Weltman-Fahs, M. 2013. Development of a 

hydrologic foundation and flow-ecology relationships for monitoring 

riverine resources in the Marcellus Shale region, phase 1 project 

report. Prepared for the Appalachian Landscape Conservation 

Cooperative, grant number 2012-03. http://applcc.org/research/

aquatic-ecological-flows-group. [Date last accessed: July 3, 2019].

Foley, J.A.; DeFries, R.; Asner, G.P. [and others]. 2005. Global 

consequences of land use. Science. 309: 570–574. DOI:0.1126/

science.1111772.

Freeman, M.C.; Buell, G.R.; Hay, L.E. [and others]. 2013. Linking river 

management to species conservation using dynamic landscape-scale 

models. River Research and Applications. 29: 906–918. DOI:10.1002/

rra.v29.7. 

Friedman, J.H. 2001. Greedy function approximation: a gradient boosting 

machine. Annals of Statistics. 29(5): 1189–1232. DOI:10.1214/

aos/1013203451. 

Fry, J.; Xian, G.; Jin, S. [and others]. 2011. Completion of the 2006 

National Land Cover Database for the conterminous United States. 

Photogrammetric Engineering & Remote Sensing. 77(9): 858–864.

Georgakakos, A.; Fleming, P.; Dettinger, M. [and others]. 2014: Water 

resources. In: Melillo, J.M.; Richmond, T.C.; Yohe, G.W., eds. Climate 

change impacts in the United States: the third National Climate 

Assessment. Washington, DC: U.S. Global Change Research Program: 

69–112. Chapter 3. DOI:10.7930/ J0G44N6T.

Georgakakos, A.; Zhang, F.; Yao, H. 2010. Climate variability and 

change assessment for the ACF River Basin, Southeast US. Georgia 

Water Resources Institute (GWRI) Technical Report sponsored by 

NOAA, USGS, and Georgia EPD. Atlanta, GA: Georgia Institute of 

Technology. 321 p.

Gerten, D.; Rost, S.; von Blo, W.; Lucht, W. 2008. Causes of change in 

20th century global river discharge. Geophysical Research Letters. 

35: L20405. DOI:10.1029/2008GL035258.

Gleeson, T.; Smith, L.; Moosdorf, N. [and others]. 2011. Mapping 

permeability over the surface of the Earth. Geophysical Research 

Letters. 38: L02401. DOI:10.1029/2010GL045565.

Graf, W.L. 1999. Dam nation: a geographic census of American dams 

and their large-scale hydrologic impacts. Water Resources Research. 

35: 1305–1311.

Hain, E.F.; Kennen, J.G.; Caldwell, P.V. [and others]. 2018. Using 

regional scale flow–ecology modeling to identify catchments where 

fish assemblages are most vulnerable to changes in water availability. 

Freshwater Biology. 63: 928–945. DOI:10.1111/fwb.13048. 
USDA Forest Service, Southern Research Station 71 e-General Technical Report SRS-246

References continued

Haith, D.A.; Mandel, R.; Wu, R.S. 1992. GWLF: Generalized Watershed 

Loading Functions user’s manual, vers. 2.0. Ithaca, NY: Cornell 

University. 62 p. 

Haith, D.A.; Shoemaker, L.L. 1987. Generalized Watershed Loading 

Functions for stream flow nutrients. Water Resources Bulletin. 

23: 471–478.

Hamon, W.R. 1963. Computation of direct runoff amounts from storm 

rainfall. International Association of Scientific Hydrology, Publication 

63: 52–62. 

Hastie, T.; Tibshirani, R.; Friedman, J.H. 2009. The elements of statistical 

learning: data mining, inference, and prediction. Springer Series in 

Statistics (2nd ed.). New York, NY: Springer-Verlag. 533 p. 

Hay, L.E.; Clark, M.P.; Wilby, R.L. [and others]. 2002. 

Use of regional climate model output for hydrologic 

simulations. Journal of Hydrometeorology. 3: 571–590. DOI: 

10.1175/1525-7541(2002)003<0571:uorcmo>2.0.co;2.

Hay, L.E.; Leavesley, G.H.; Clark, M.P. [and others]. 2006. Step 

wise, multiple objective calibration of a hydrologic model for a 

snowmelt dominated basin. Journal of the American Water Resources 

Association. 42: 877–890. DOI: 10.1111/j.1752-1688.2006.tb04501.x.

Hay, L.E.; McCabe, G.J. 2002. Spatial variability in water-balance 

model performance in the conterminous United States. Journal 

of the American Water Resources Association. 38: 847–860. 

DOI:10.1111/j.1752-1688.2002.tb01001.x.

Hay, L.E.; Umemoto, M. 2006. Multiple-objective stepwise calibration 

using Luca. Open-File Report 2006–1323. Reston, VA: U.S. 

Department of the Interior, U.S. Geological Survey. 25 p.

Henriksen, J.A.; Heasley, J.; Kennen, J.G.; Nieswand, S. 2006. Users’ 

manual for the Hydroecological Integrity Assessment Process software 

(including the New Jersey Assessment Tools). Open-File Report 

2006–1093. https://pubs.er.usgs.gov/publication/ofr20061093. [Date 

last accessed: July 3, 2019].

Hijmans, R.J. 2012. Cross-validation of species distribution models: 

removing spatial sorting bias and calibration with a null model. 

Ecology. 93(3): 679–688. DOI:10.1890/11-0826.1. 

Hill, M.T.; Platts, W.S.; Beschta, R.L. 1991. Ecological and 

geomorphological concepts for instream and out-of-channel flow 

requirements. Rivers. 2: 198–210. 

Homer, C.; Dewitz, J.; Fry, J. [and others]. 2007. Completion of the 2001 

National Land Cover Database for the conterminous United States. 

Photogrammetric Engineering & Remote Sensing. 73: 337–341.

Intergovernmental Panel on Climate Change (IPCC). 2013. Climate 

change 2013: the physical science basis. Contribution of Working 

Group I to the Fifth Assessment Report of the Intergovernmental 

Panel on Climate Change. Stocker, T.F.; Qin, D.; Plattner, G.-K. 

[and others], eds. Cambridge, U.K. and New York, NY: Cambridge 

University Press. 1535 p. 

Jackson, R.B.; Carpenter, S.R.; Dahm, C.N. [and others]. 2001. Water 

in a changing world. Ecological Applications. 11: 1027–1045. DOI: 

10.2307/3061010.

James, G.; Witten, D.; Hastie, T.; Tibshirani, R. 2013. An introduction to 

statistical learning. New York, NY: Springer. 426 p. 

Jensen, M.E.; Burman, R.D.; Allen, R.G. 1990. Evapotranspiration and 

irrigation water requirements. New York, NY: ASCE. 360 p. 

Jensen, M.E.; Haise, H.R. 1963. Estimating evapotranspiration from solar 

radiation. Journal of Irrigation and Drainage Division. 89: 15–41.

Jha, M.; Arnold, J.G.; Gassman, P.W. [and others]. 2006. Climate change 

sensitivity assessment on upper Mississippi River Basin streamflows 

using SWAT. Journal of the American Water Resources Association. 

42 : 997–1015. DOI: 10.1111/j.1752-1688.2006.tb04510.x.

Johnson, T.; Butcher, J.; Weaver, C. 2012. Investigating the implications 

of climate change for U.S. stream water quality: the EPA Global 

Change Research Program’s “20 Watersheds” project. Journal of 

Water Resources Planning and Management. 138(5): 453–464.

Keaton, M.; Haney, D.; Andersen, C.B. 2005. Impact of drought upon 

fish assemblage structure in two South Carolina Piedmont streams. 

Hydrobiologia. 54: 209–223. DOI:10.1007/s10750-005-2674-z.

Kendy, E.; Apse, C.; Blann, K. 2012. A practical guide to 

environmental flows for policy and planning. Arlington, VA: The 

Nature Conservancy. 72 p. https://www.conservationgateway.

org/ConservationByGeography/NorthAmerica/UnitedStates/edc/

Documents/ED_freshwater_envflows_Practical%20Guide%20

Eflows%20for%20Policy.pdf. [Date last accessed: October 1, 2019].

Kendy, E.; Cutrofello, M.; Fekadu, M. 2011. Unified hydrologic model 

for assessing human and climate impacts on streamflows at multiple 

geographic scales. http://data.southatlanticlcc.org/TNC_flows_for_

web.pdf. [Date accessed: August 23, 2017].

Kennard, M.J.; Pusey, B.J.; Olden, J.D. [and others]. 2010. Classification 

of natural flow regimes in Australia to support environmental flow 

management. Freshwater Biology. 55: 171–193. DOI:10.1111/

fwb.2009.55.issue-1.

Kennen, J.G.; Chang, M.; Tracy, B.H. 2005. Effects of landscape change 

on fish assemblage structure in a rapidly growing metropolitan area 

in North Carolina, USA. In: Brown, L.R.; Hughes, R.M.; Gray, R.; 

Meador, M.R., eds. Effects of urbanization on stream ecosystems. 

American Fisheries Society symposium 47. Bethesda, MD: American 

Fisheries Society: 39–52.

Kennen, J.G.; Henriksen, J.A.; Heasley, J. [and others]. 2009. Application 

of the Hydroecological Integrity Assessment Process for Missouri 

streams. Open-File Report 2006–1093. 71 p. https://pubs.er.usgs.gov/

publication/ofr20091138. [Date last accessed: July 3, 2019].

Kennen, J.G.; Henriksen, J.A.; Nieswand, S.P. 2007. Development of 

the Hydroecological Integrity Assessment Process for determining 

environmental flows for New Jersey streams. U.S. Geological 

Survey Scientific Investigations Report 2007-5206. http://pubs.usgs.

gov/sir/2007/5206/pdf/sir2007-5206-508.pdf. [Date last accessed: 

July 3, 2019].

Kennen, J.G.; Riskin, M.L.; Charles, E.G. 2014. Effects of streamflow 

reductions on aquatic macroinvertebrates: linking groundwater 

withdrawals and assemblage response in southern New Jersey streams, 

USA. Hydrological Sciences Journal. 59: 545–561. DOI:10.1080/026

26667.2013.877139. 

Kennen, J.G.; Riva-Murray, K.; Beaulieu, K.M. 2010. Determining 

hydrologic factors that influence stream macroinvertebrate 

assemblages in the northeastern US. Ecohydrology. 3: 88–106. 

DOI:10.1002/eco.99. 

King, R.S.; Baker, M.E.; Whigham, D.F. [and others]. 2005. Spatial 

consideration for linking watershed land cover to ecological indicators 

in streams. Ecological Applications. 15: 137-153. DOI:10.1890/04-

0481.

Knight, R.R.; Gain, W.S.; Wolfe, W.J. 2012. Modelling ecological flow 

regime: an example from the Tennessee and Cumberland River Basins. 

Ecohydrology. 5: 613–627. DOI:10.1002/eco.246.

Knight, R.R.; Murphy, J.C.; Wolfe, W.J. [and others]. 2014. Ecological 

limit functions relating fish community response to hydrologic 

departures of the ecological flow regime in the Tennessee River Basin, 

United States. Ecohydrology. 7: 1262– 1280. DOI:10.1002/eco.1460.

Konrad, C.P.; Brasher, A.M.D.; May, J.T. 2008. Assessing streamflow 

characteristics as limiting factors on benthic invertebrate assemblages 

in streams across the Western United States. Freshwater Biology. 

53: 1983–1998. DOI:10.1111/j.1365-2427.2008.02024.x.
72

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Koren. V.; Smith, M.; Duan, Q. 2003. Use of a priori parameter estimates 

in the derivation of spatially consistent parameter sets of rainfallrunoff models. In: Duan, Q.; Gupta, H.; Sorooshian, S. [and others], 

eds. Calibration of watershed models, volume 6. Washington, D.C. 

239–254. DOI:10.1002/9781118665671.ch18.

LaFontaine, J.H.; Hay, L.E.; Viger, R.J. [and others]. 2013. Application 

of the Precipitation-Runoff Modeling System (PRMS) in the 

Apalachicola–Chattahoochee–Flint River Basin in the Southeastern 

United States. U.S. Geological Survey Scientific Investigations 

Report 2013–5162. Reston, VA: U.S. Department of the Interior, 

U.S. Geological Survey. 118 p.

Leavesley, G.H.; Lichty, R.W.; Troutman, B.M.; Saindon, L.G. 1983. 

Precipitation-Runoff Modeling System—user’s manual. U.S. 

Geological Survey Water-Resources Investigations Report 83–4238. 

Denver, CO: U.S Department of the Interior, U.S. Geological Survey. 

207 p.

Leclere, J.; Oberdorff, T.; Belliard, J.; Leprieur, F. 2011. A comparison of 

modeling techniques to predict juvenile 0+ fish species occurrences 

in a large river system. Ecological Informatics. 6: 276–285. 

DOI:10.1016/j.ecoinf.2011.05.001.

Legesse, D.; Abiye, T.A.; Vallet-Coulomb, C.; Abate, H. 2010. 

Streamflow sensitivity to climate and land cover changes: Meki River, 

Ethiopia. Hydrology and Earth System Sciences. 14: 2277–2287. 

DOI:10.5194/hess-14-2277-2010.

Lockaby, G.; Nagy, C.; Vose, J.M. [and others]. 2013. Forests and water. 

In: Wear, D.N.; Greis, J.G., eds. The Southern Forest Futures Project: 

technical report. Gen. Tech. Rep. SRS-178. Asheville, NC: U.S. 

Department of Agriculture Forest Service, Southern Research Station: 

309–339.

Lumb, A.M.; McCammon, R.B.; Kittle, J.L., Jr. 1994. User’s manual 

for an expert system (HSPEXP) for calibration of the Hydrological 

Simulation Program-Fortran. U.S. Geological Survey Water Resources 

Investigation Report 94-4168. Reston, VA: U.S. Department of the 

Interior, U.S. Geological Survey. 102 p.

Lytle, D.A.; Poff, N.L. 2004. Adaptation to natural flow regimes. Trends 

in Ecology & Evolution. 19: 94–100. DOI:10.1016/j.tree.2003.10.002. 

Marion, D.A.; Sun, G.; Caldwell, P.V. [and others]. 2013. Managing 

forest water quantity and quality under climate change. In: Vose, 

J.; Klepzig, K.D., ed. Climate change adaptation and mitigation 

management options: a guide for natural resource managers in 

southern forest ecosystems. Boca Raton, FL: CRC Press. 492 p.

Markstrom, S.L.; Niswonger, R.G.; Regan, R.S. [and others]. 2008. 

GSFLOW—coupled groundwater and surface-water flow model 

based on the integration of the Precipitation-Runoff Modeling 

System (PRMS) and the Modular Ground-Water Flow Model 

(MODFLOW–2005). In: U.S. Geological Survey techniques and 

methods, book 6. 240 p. 

Masters, L.L.; Flack, S.R.; Stein, B.A. 1998. Rivers of life: critical 

watersheds for protecting freshwater biodiversity. Arlington, VA: 

The Nature Conservancy. https://www.natureserve.org/biodiversityscience/publications/rivers-life-critical-watersheds-protectingfreshwater-biodiversity. [Date last accessed: July 3, 2019].

Matthews, R. 2005. A six-step framework for ecologically sustainable 

water management. Journal of Contemporary Water Research 

and Education. 131: 60–65. DOI:10.1111/j.1936-704X.2005.

mp131001010.x.

Matthews, W.J.; Marsh-Matthews, E. 2003. Effects of drought on fish 

across axes of space, time and ecological complexity. Freshwater 

Biology. 48: 1232–1253. DOI:10.1046/j.1365-2427.2003.01087.x. 

Maupin, M.A.; Kenny, J.F.; Hutson, S.S. [and others]. 2014. Estimated 

use of water in the United States in 2010. U.S. Geological Survey 

Circular 1405. Reston, VA: U.S. Department of the Interior, U.S. 

Geological Survey. 56 p. https://pubs.usgs.gov/circ/1405/pdf/circ1405.

pdf. [Date last accessed: July 3, 2019].

Maurer, E.P.; Wood; A.W.; Adam, J.C. [and others]. 2002. A long-term 

hydrologically-based data set of land surface fluxes and states for the 

conterminous United States. Journal of Climate. 15: 3237–3251.

McCabe, G.J.; Markstrom, S.L. 2007. A monthly water-balance model 

driven by a graphical user interface. Open-File Report 2007–1088. 

Reston, VA: U.S. Department of the Interior, U.S. Geological Survey. 

6 p.

McCabe, G.J.; Wolock, D.M. 2011. Independent effects of temperature 

and precipitation on modeled runoff in the conterminous 

United States. Water Resources Research. 47: W11522. DOI: 

10.1029/2011WR010630.

McManamay, R.A.; Orth, D.J.; Dolloff, C.A.; Frimpong, E.A. 2012. A 

regional classification of unregulated streamflows: spatial resolution 

and hierarchical frameworks. River Research and Applications. 

28: 1019–1033. DOI:10.1002/rra.v28.7. 

McManamay, R.A.; Orth, D.J.; Dolloff, C.A.; Mathews, D.C. 2013. 

Application of the ELOHA framework to regulated rivers in the Upper 

Tennessee River Basin: a case study. Environmental Management. 

51: 1210–1235. DOI:10.1007/s00267-013-0055-3.

Meehl, G.; Covey, C.; Delworth, T. [and others]. 2007. The WCRP 

CMIP3 multi-model dataset: a new era in climate change research. 

Bulletin of the American Meteorological Society. 88: 1383–1394. 

DOI:10.1175/BAMS-88-9-1383. 

Melillo, J.M.; Richmond, T.C.; Yohe, G.W., eds. 2014. Climate change 

impacts in the United States: the third National Climate Assessment. 

Washington, D.C.: U.S. Global Change Research Program. 841 p. 

DOI:10.7930/j0z31wj2. 

Mengistu D.T. and Sorteberg A. 2012. Sensitivity of SWAT simulated 

streamflow to climatic changes within the Eastern Nile River Basin. 

Hydrology and Earth System Sciences. 16: 391– 407. doi:10.5194/

hess-16-391-2012.

Milly, P.C.D.; Betancourt, J.; Falkenmark, M. [and others]. 2008. 

Stationarity is dead: whither water management? Science. 

319: 573– 574. DOI:10.1126/science.1151915.

Mims, M.C.; Olden, J.D. 2012. Life history theory predicts fish 

assemblage response to hydrologic regimes. Ecology. 93: 35–45. 

DOI:10.1890/11-0370.1.

Mims, M.C.; Olden, J.D. 2013. Fish assemblages respond to altered flow 

regimes via ecological filtering of life history strategies. Freshwater 

Biology. 58: 50–62. DOI:10.1111/fwb.12037.

Monk, W.A.; Wood; P.J.; Hannah; D.M.; Wilson, D.A. 2007. Selection of 

river flow indices for the assessment of hydroecological change. River 

Research and Applications. 23: 113–122. DOI:10.1002/(ISSN)1535-

1467.

Monteith, J.L. 1965. Evaporation and the environment in the state and 

movement of water in living organisms. In: Proceedings of the Society 

for Experimental Biology, XIXth symposium; Swansea. Cambridge, 

U.K.: Cambridge University Press: 205–234.

Moriasi, D.N.; Arnold, J.G.; Van Liew, M.W. [and others]. 2007. 

Model evaluation guidelines for systematic quantification of 

accuracy in watershed simulations. Transactions of the American 

Society of Agricultural and Biological Engineers. 50(3): 885–900. 

DOI:10.13031/2013.23153. 
USDA Forest Service, Southern Research Station 73 e-General Technical Report SRS-246

References continued

Murphy, J.C.; Knight, R.R.; Wolfe, W.J.; Gain, W.S. 2013. Predicting 

ecological flow regime at ungaged sites: a comparison of methods. 

River Research and Applications. 29: 660–669. DOI: 10.1002/

rra.2570.

Naiman, R.J.; Latterell, J.J.; Pettit, N.E.; Olden. J.D. 2008. Flow 

variability and the biophysical vitality of river systems. Comptes 

Rendus Geoscience. 340: 629–643. DOI:10.1016/j.crte.2008.01.002. 

Nakicenovic, N.; Alcamo, J.; Davis, G. [and others]. 2000. Special report 

on emissions scenarios: a special report of Working Group III of 

the Intergovernmental Panel on Climate Change. Cambridge, U.K.: 

Cambridge University Press. 599 p. 

Nash, J.E.; Sutcliffe, J.V. 1970. River flow forecasting through conceptual 

models, I, A discussion of principles. Journal of Hydrology. 

10: 282– 290. DOI:10.1016/0022-1694(70)90255-6.

NC Floodplain Mapping Program. 2013. Digital elevation model 

of North Carolina. Retrieved from https://flood.nc.gov/ncflood/

mappingprogram.html. [Date last accessed: August 15, 2019].

Neitsch, S.L.; Arnold, J.G.; Kiniry, J.R. 2005. Soil and water assessment 

tool, theoretical documentation. Temple, TX: U.S. Department of 

Agriculture Agricultural Research Service, Grassland, Soil and Water 

Research Laboratory. 476 p.

North Carolina Climate Office. 2016. Normal monthly precipitation 

totals based on 1971–2000 normals. http://climate.ncsu.edu/climate/

monthlyprecip.html. [Date accessed: August 23, 2017]. 

North Carolina Department of Environment and Natural Resources 

[NCDENR]. 2006. Standard operating procedure biological 

monitoring, stream fish community assessment program. Version 4. 

Raleigh, NC: North Carolina Department of Environment and Natural 

Resources, Division of Water Quality. 51 p. 

North Carolina Ecological Flows Scientific Advisory Board [NCEFSAB]. 

2013. Recommendations for estimating flows to maintain ecological 

integrity in streams and rivers in North Carolina. https://www.fws.

gov/asheville/pdfs/Recommendations_for_Maintaining_Flows_

FINAL%202013-10-30.pdf. [Date accessed: August 23, 2017].

Novak, R.; Kennen, J.G.; Abele, R.W. [and others]. 2016. Final 

EPA-USGS technical report: protecting aquatic life from effects of 

hydrologic alteration. U.S. Geological Survey Scientific Investigations 

Report 2016–5164, U.S. Environmental Protection Agency EPA–

822–R–156–007. 58 p. https://www.epa.gov/wqc/final-epausgstechnical-report-protecting-aquatic-life-effects-hydrologic-alterationdocuments. [Date accessed: January 31, 2018].

Olden, J.D. 2016. Challenges and opportunities for fish conservation 

in dam-impacted waters. In: Closs, G.P.; Krkosek, M.; Olden, J.D., 

eds. Conservation of freshwater fishes. Cambridge, U.K.: Cambridge 

University Press: 107–148. 

Olden, J.D.; Poff N.L. 2003. Redundancy and the choice of hydrologic 

indices for characterizing streamflow regimes. River Research and 

Applications. 19: 101–121. DOI: 10.1002/rra.700.

Piñeiro, G.; Perelman, S.; Guerschman, J.P.; Paruelo, J.M. 2008. 

How to evaluate models: observed vs. predicted or predicted vs. 

observed? Ecological Modelling. 216: 316–322. DOI:10.1016/j.

ecolmodel.2008.05.006.

Poff, N.L. 2018. Beyond the natural flow regime? Broadening the 

hydro-ecological foundation of environmental flows science in 

a non-stationary world. Freshwater Biology. 63(8): 1011–1021. 

DOI:10.1111/fwb.13038.

Poff, N.L.; Allan, J.D.; Bain, M.B. [and others]. 1997. The natural 

flow regime—a paradigm for river conservation and restoration. 

BioScience. 47: 769–784. DOI: 10.2307/1313099.

Poff, N.L.; Olden, J.D.; Merritt, D.M.; Pepin, D.M. 2007. 

Homogenization of regional river dynamics by dams and global 

biodiversity implications. Proceeding of the National Academy of 

Science. 104: 5732−5737. DOI: 10.1073/pnas.0609812104.

Poff, N.L.; Richter, B.D.; Arthington, A.H. [and others]. 2010. 

The ecological limits of hydrologic alteration (ELOHA)—a 

new framework for developing regional environmental flow 

standards. Freshwater Biology. 55: 147–170. DOI:10.1111/j.1365-

2427.2009.02204.x.

Poff, N.L.; Ward, J.V. 1989. Implications of streamflow variability and 

predictability for lotic community structure: a regional analysis of 

streamflow patterns. Canadian Journal of Fisheries and Aquatic 

Sciences. 46: 1805–1818. DOI:10.1139/f89-228.

Poff, N.L.; Zimmerman, J.K.H. 2010. Ecological responses to 

altered flow regimes: a literature review to inform the science 

and management of environmental flows. Freshwater Biology. 

55: 194– 205. DOI: 10.1111/j.1365-2427.2009.02272.x.

Postel, S.; Richter, B. 2012. Rivers for life: managing water for people 

and nature. Washington, D.C.: Island Press. 253 p.

Prasad, A.M.; Iverson, L.R.; Liaw, A. 2006. Newer classification and 

regression tree techniques: bagging and random forests for ecological 

prediction. Ecosystems. 9: 181–199. DOI:10.1007/s10021-005-

0054- 1. 

Price, C.V.; Nakagaki, N.; Hitt, K.J.; Clawges, R.M. 2006. Enhanced 

historical land-use and land-cover data sets of the U.S. Geological 

Survey. U.S. Department of the Interior, U.S. Geological 

Survey. https://pubs.usgs.gov/ds/2006/240/. [Date last accessed: 

August 15, 2019].

PRISM (Parameter-elevation Relationships on Independent Slopes 

Model) Climate Group. 2013. PRISM climate data. http://www.prism.

oregonstate.edu/. [Date accessed: August 23, 2017].

Randin, C.F.; Dirnbock, T.; Dullinger, S. [and others]. 2006. Are nichebased species distribution models transferable in space? Journal of 

Biogeography. 33: 1689–1703. DOI:10.1111/jbi.2006.33.issue-10. 

Raupach, M.R.; Marland, G.; Ciais, P. [and others]. 2007. Global and 

regional drivers of accelerating CO2 emissions. Proceedings of the 

National Academy of Sciences of the United States of America. 

104: 10288–10293. DOI:10.1073/pnas.0700609104. 

Richter, B.D.; Baumgartner, J.V.; Wigington, R.; Braun, D.P. 1997. How 

much water does a river need? Freshwater Biology. 37: 231-249. DOI: 

10.1046/j.1365-2427.1997.00153.x.

Richter, B.D.; Braun, D.P.; Mendelson, M.A.; Master, L.L. 1997. Threats 

to imperiled freshwater fauna. Conservation Biology. 11: 1081–1093. 

DOI:10.1046/j.1523-1739.1997.96236.x. 

Sankarasubramanian, A.; Vogel, R.M. 2001. Climate elasticity of 

streamflow in the United States. Water Resources Research. 

37: 1771– 1781. DOI: 10.1029/2000WR900330.

SAS Institute Inc. 1989. SAS/STAT Users Guide. Version 6, vol. 2, 4th 

edition. Cary, NC: SAS Institute Inc.

Seager, D.; Tzanova, A.; Nakamura, J. 2009. Drought in the Southeastern 

United States, variability over the last millennium, and the potential 

for future hydroclimate change. Journal of Climate. 22: 5021–5045. 

DOI:10.1175/2009JCLI2683.1. 

Shenton, W.; Bond, N.R.; Yen, J.D.; Mac Nally, R. 2012. Putting the 

“ecology” into environmental flows: ecological dynamics and 

demographic modelling. Environmental Management. 50: 1–10. 

DOI:10.1007/s00267-012-9864-z.
74

Hydrologic Modeling for Flow-Ecology Science in the Southeastern United States and Puerto Rico

Sivapalan, M.; Takeuchi, K.; Franks, S.W. [and others]. 2003. IAHS 

decade on predictions in ungauged basins (PUB), 2003–2012: shaping 

an exciting future for the hydrological sciences. Hydrological Sciences 

Journal. 48(6): 857–880. DOI:10.1623/hysj.48.6.857.51421. 

South Atlantic Landscape Conservation Cooperative [SALCC]. 2012. 

South Atlantic Landscape Conservation Cooperative science 

assessment 2012. http://www.southatlanticlcc.org/page/scienceassessment. [Date accessed: August 23, 2017].

Southeast Aquatic Resources Partnership [SARP]. 2004. Conserving our 

Southeastern aquatic habitat: the Southeast aquatic habitat plan. http://

southeastaquatics.net/about/our-work/conservation-planning/sahp/

sahp. [Date accessed: August 23, 2017].

Stalnaker, C.B. 1990. Minimum flow is a myth. In: Bain, M.B., ed. 

Ecology and assessment of warmwater streams: workshop synopsis. 

Biological Report. Washington, D.C.: U.S. Fish & Wildlife Service.: 

31–33. 

Stanford, J.A.; Ward, J.V.; Liss, W.J. [and others]. 1996. A general 

protocol for restoration of regulated rivers. River Research and 

Applications. 12: 391–413. DOI:391-414. 10.1002/(SICI)1099-

1646(199607)12:4/5<391::AID-RRR436>3.0.CO;2-4.

Statzner, B.; Hildrew, A.G.; Resh, V.H. 2001. Species traits and 

environmental constraints: entomological research and the history 

of ecological theory. Annual Review of Entomology. 46: 291–316. 

DOI:10.1146/annurev.ento.46.1.291. 

Stewart-Koster, B.; Olden, J.D.; Gido, K.B. 2014. Quantifying flow–

ecology relationships with functional linear models. Hydrological 

Sciences Journal. 59: 629–644. DOI:10.1080/02626667.2013.860231. 

Sun, G.; Alstad, K.; Chen, J. [and others]. 2011a. A general predictive 

model for estimating monthly ecosystem evapotranspiration. 

Ecohydrology. 4: 245–255. DOI:10.1002/eco.194.

Sun, G.; Caldwell, P.; Noormets, A. [and others]. 2011b. Upscaling 

key ecosystem functions across the conterminous United States by 

a water-centric ecosystem model. Journal of Geophysical Research. 

116: G00J05. DOI:10.1029/2010JG001573.

Sun, G.; Caldwell, P.V.; Georgakakos, A.P. [and others]. 2013. Impacts of 

climate change and variability on water resources in the Southeastern 

US. In: Ingram, K.T.; Dow K.; Carter, L., eds. Southeast regional 

technical report to the National Climate Change Assessment, water 

resources. Washington, D.C.: Island Press: 210–236.

Sun, G.; McNulty, S.G.; Moore Myers, J.A.; Cohen, E.C. 2008. 

Impacts of multiple stresses on water demand and supply across the 

Southeastern United States. Journal of the American Water Resources 

Association. 44: 1441–1457. DOI:10.1111/j.1752-1688.2008.00250.x.

Tang, C.; Crosby, B.T.; Wheaton, J.M.; Piechota, T.C. 2012. Assessing 

streamflow sensitivity to temperature increases in the Salmon River 

Basin, Idaho. Global and Planetary Change. 88–89: 32–44. 

Tavernia, B.G.; Nelson, M.D.; Caldwell, P.; Sun, G. 2013. Water stress 

projections in the Northeastern and Midwestern United States in 2060: 

anthropogenic and ecological consequences. Journal of the American 

Water Resources Association. 49: 938–952. DOI: 10.1111/jawr.12075.

Thornthwaite, C.W. 1948. An approach toward a rational classification of 

climate. Geographical Review. 38: 55–94.

Thornton, P.E.; Thornton, M.M.; Mayer, B.W. [and others]. 2013. 

Daymet: daily surface weather on a 1 km grid for North America, 

1980–2012. http://daymet.ornl.gov/. [Date accessed: July, 2013].

Treasure, E.; McNulty, S.; Moore Myers, J.; Jennings, L.N. 2014. 

Template for assessing climate change impacts and management 

options: TACCIMO user guide version 2.2. Gen. Tech. Rep. SRS186. Asheville, NC: U.S. Department of Agriculture Forest Service, 

Southern Research Station. 33 p.

Turner, M.; Stewardson, M. 2014. Hydrologic indicators of hydraulic 

conditions that drive flow–biota relationships. Hydrological Sciences 

Journal. 59: 659–672. DOI:10.1080/02626667.2014.896997.

U.S. Census Bureau. 2016. United States census 2010. http://www.

census.gov/2010census/. [Date last accessed: July 5, 2019]. 

U.S. Department of Agriculture Natural Resources Conservation Service 

[USDA NRCS]. 1986. Urban hydrology for small watersheds. 

Technical Release 55 (TR-55). Washington, DC: U.S. Department 

of Agriculture Soil Conservation Service, Conservation Engineering 

Division. 164 p.

U.S. Department of Agriculture Natural Resources Conservation 

Service [USDA NRCS]. 2012. U.S. general soil map (STATSGO2). 

https://www.nrcs.usda.gov/wps/portal/nrcs/detail/soils/survey/

geo/?cid=nrcs142p2_053629. [Date accessed: August 23, 2017].

U.S. Department of Agriculture Soil Conservation Service. 1972. 

Hydrology guide for use in watershed planning. National engineering 

handbook, section 4: hydrology, supplement A. Washington, D.C.: 

U.S. Department of Agriculture, Natural Resources Conservation 

Service. 127 p. 

U.S. Environmental Protection Agency [USEPA]. 2008. Using the 

BASINS meteorological database―version 2006. Washington, D.C.: 

U.S. Environmental Protection Agency Office of Water. 6 p.

U.S. Environmental Protection Agency [USEPA]. 2009. Land-use 

scenarios: national-scale housing-density scenarios consistent with 

climate change storylines. EPA/600/R-08/076F. Washington, D.C.: 

Global Change Research Program, National Center for Environmental 

Assessment. 137 p. 

U.S. Environmental Protection Agency [USEPA]. 2011. Healthy 

watersheds initiative: national framework and action plan. 

Washington, D.C.: U.S. Environmental Protection Agency Office of 

Water. https://www.epa.gov/sites/production/files/2015-10/documents/

hwi_action_plan.pdf. [Date last accessed: July 5, 2019].

U.S. Environmental Protection Agency [USEPA]. 2012. United States 

Environmental Protection Agency: Safe Drinking Water Information 

System. October 2012. Washington, D.C.: U.S. Environmental 

Protection Agency. https://www.epa.gov/ground-water-and-drinkingwater/safe-drinking-water-information-system-sdwis-federalreporting. [Date accessed: November, 2012].

U.S. Environmental Protection Agency [USEPA]. 2013. Watershed 

modeling to assess the sensitivity of streamflow, nutrient, and 

sediment loads to potential climate change and urban development 

in 20 U.S. watersheds. Washington, D.C.: National Technical 

Information Service:https://cfpub.epa.gov/ncea/global/recordisplay.

cfm?deid=256912. [Date last accessed: July 5, 2019].

Van Sickle, J. 2003. Analyzing correlations between stream and 

watershed attributes. Journal of the American Water Resources 

Association. 39: 717–726. DOI: 10.1111/j.1752-1688.2003.tb03687.x.

Verberk, W.; Van Noordwijk, C.; Hildrew, A. 2013. Delivering on a 

promise: integrating species traits to transform descriptive community 

ecology into a predictive science. Freshwater Science. 32(2): 531–547. 

DOI:10.1899/12-092.1. 

Violle, C.; Navas, M.L.; Vile, D. [and others]. 2007. Let the concept of 

trait be functional! Oikos. 116: 882–892. https://onlinelibrary.wiley.

com/doi/10.1111/j.0030-1299.2007.15559.x. 

Vogelmann, J.E.; Howard, S.M.; Yang, L. [and others]. 2001. Completion 

of the 1990’s National Land Cover Data Set for the conterminous 

United States. Photogrammetric Engineering and Remote Sensing. 

67: 650-662.

Vörösmarty, C.J.; McIntyre, P.B.; Gessner, M.O. [and others]. 2010. 

Global threats to human water security and river biodiversity. Nature. 

467: 555–561. DOI:10.1038/nature09440. 
USDA Forest Service, Southern Research Station 75 e-General Technical Report SRS-246

References continued

Waite, I.R.; Brown, L.R.; Kennen, J.G. [and others]. 2010. Comparison 

of watershed disturbance predictive models for stream benthic 

macroinvertebrates for three distinct ecoregions in Western 

US. Ecological Indicators. 10: 1125–1136. DOI:10.1016/j.

ecolind.2010.03.011. 

Waite, I.R.; Kennen, J.G.; May, J.T. [and others]. 2012. Comparison of 

stream invertebrate response models for bioassessment metrics. PLoS 

ONE. 9(3): e90944. DOI:10.1371/journal.pone.0090944. 

Walsh, J.; Wuebbles, D.; Hayhoe, K. [and others]. 2014. Our changing 

climate. In: Melillo, J.M.; Richmond, T.C.; Yohe, G.W., eds. Climate 

change impacts in the United States: the third National Climate 

Assessment. Washington, D.C.: U.S. Global Change Research 

Program: 19–67. Chapter 2. DOI:10.7930/J0KW5CXT.

Warren, M.L., Jr.; Burr, B.M.; Walsh, S.J. [and others]. 2000. Diversity, 

distribution, and conservation status of the native freshwater fishes of 

the Southern United States. Fisheries. 25: 7–29. DOI:10.1577/1548-

8446(2000)025<0007:DDACSO>2.0.CO;2.

Welch, B.L. 1947. The generalization of student’s’ problem when several 

different population variances are involved. Biometrika. 34: 28–35. 

DOI:10.2307/2332510. 

Wen, M.; Zhang, Y. 2009. Agricultural water use estimation for the Flint 

River Basin Regional Water Development and Conservation Plan. 

In: Proceedings of the 2009 Georgia Water Resources Conference. 

Athens, GA: University of Georgia: 622–625.

Wenger, S.J.; Luce, C.H.; Hamlet, A.F. [and others]. 2010. Macroscale 

hydrologic modeling of ecologically relevant flow metrics. Water 

Resources Research. 46: W09513. DOI: 10.1029/2009WR008839.

Wheeler, K.; Wenger, S.; Freeman, M. 2018. States and rates: 

complementary approaches to developing flow-ecology relationships. 

Freshwater Biology. 63: 906–916. DOI:10.1111/fwb.13001. 

White, R.S.A.; McHugh, P.A.; McIntosh, A.R. 2016. Drought survival 

is a threshold function of habitat size and population density in 

a fish metapopulation. Global Change Biology. 22: 3341–3348. 

DOI:10.1111/gcb.2016.22.issue-10. 

Williamson, T.N.; Odom, K.R.; Newson, J.K. [and others]. 2009. The 

Water Availability Tool for Environmental Resources (WATER)—a 

water-budget modeling approach for managing water-supply resources 

in Kentucky—phase I—data processing, model development, and 

application to non-karst areas, U.S. Geological Survey Scientific 

Investigations Report 2009-5248. Reston, VA: U.S. Department 

of the Interior, U.S. Geological Survey. 34 p. http://pubs.usgs.gov/

sir/2009/5248. [Date accessed: August 23, 2017].

Williamson, T.N.; Taylor, C.J.; Newson, J.K. 2013. Significance of 

exchanging SSURGO and STATSGO data when modeling hydrology 

in diverse physiographic terranes. Soil Science Society of America 

Journal. 77: 877–889. DOI: 10.2136/sssaj2012.0069.

Wolock, D.M. 1993. Simulating the variable-source-area concept of 

streamflow generation with the watershed model TOPMODEL. U.S. 

Geological Survey Water Resources Investigation Report 93-4124. 

Lawrence, KS: U.S. Department of the Interior, U.S. Geological 

Survey. 33 p.

Worrall, T.P.; Dunbar, M.J.; Extence, C.A. [and others]. 2014. The 

identification of hydrological indices for the characterization of 

macroinvertebrate community response to flow regime variability. 

Hydrological Sciences Journal. 59: 645–658. DOI:10.1080/02626667

.2013.825722. 

Zhang, L.; Sun, G.; Cohen, E. [and others]. 2018. An improved 

water budget for the El Yunque National Forest, Puerto Rico, as 

determined by the Water Supply Stress Index Model. Forest Science. 

64(3): 268– 279. https://doi.org/10.1093/forsci/fxx016.

Zhao, M.; Heinsch, F.A.; Nemani, R.R.; Running, S.W. 2005. 

Improvements of the MODIS terrestrial gross and net primary 

production global data set. Remote Sensing of Environment. 

95: 164– 176. DOI:10.1016/j.rse.2004.12.011.
76

List of Acronyms and Abbreviations

ACF Apalachicola-Chattahoochee-Flint

AGWRC Groundwater recession rate

AMJCMS Average streamflow for April, May, and June (spring flow) in m3 s-1

AugCMS Average streamflow for August (summer flow) in m3 s-1

AveCMS Average monthly streamflow in m3 s-1

AWC Available water capacity

BASETP Evapotranspiration by riparian vegetation

BAU Biological Assessment Unit

BRT Boosted regression tree

CANMAX Maximum canopy storage

CART Classification and regression tree

CFS Cubic feet per second

CFSM Cubic feet per second per square mile

CRAM A proprietary network flow model used to simulate water resources systems

DAYMET Daily Meteorological Data

ELOHA Ecological Limits of Hydrologic Alteration

ERFM Ecologically relevant flow metrics

ET Evapotranspiration

FloVar Measure of the coefficient of monthly streamflow variability

FSR Fish species richness

GCM General Circulation Model

GIRAS Geographic Information Retrieval and Analysis System

GIS Geographic Information System

GR4 A lumped bucket-type daily rainfall-runoff model

GW_DELAY Groundwater delay time

GWLF Generalized Watershed Loading Function

GWQMN Threshold depth of water in the shallow aquifer required for return flow to occur

HRU Hydrologic Response Unit

HSG Hydrologic soil group

HSPF Hydrological Simulation Program-Fortran

HUC Hydrologic Unit Code

IDW Inverse Distance Weighted

INFILT Index to mean soil infiltration rate

IPCC Intergovernmental Panel on Climate Change

LAI Leaf area index

LCC Landscape Conservation Cooperative

LZSN Lower zone nominal soil moisture storage

MaxCMS Maximum monthly streamflow in m3 s-1

MinCMS Minimum monthly streamflow in m3 s-1

MODIS Moderate Resolution Imaging Spectroradiometer

MWBM Monthly Water Balance Model

NATHAT National Hydrologic Assessment Tool

NCDWR North Carolina Division of Water Resources
USDA Forest Service, Southern Research Station 77 e-General Technical Report SRS-246

List of Acronyms and Abbreviations continued

NCEFSAB North Carolina Ecological Flows Scientific Advisory Board

NEXRAD Next-Generation Radar

NFRP Natural Flow Regime Paradigm

NHD National Hydrography Dataset

NLCD National Land Cover Database

NRCS Natural Resources Conservation Service, U.S. Department of Agriculture

NSE Nash-Sutcliffe Efficiency

Pbias Monthly precipitation bias

PDP Partial dependency plot

PEST Parameter Estimation Tool

PET Potential evapotranspiration

PETfac A potential evapotranspiration correction factor

PPT Precipitation

PRISM Parameter-elevation Regressions on Independent Slopes Model

PRMS Precipitation-Runoff Modeling System

QFRAC_IMP Proportion of surface flow coming from impervious surfaces

RCoeff Recession coefficient providing the rate of release from the saturated zone to the stream channel

RevapMN Threshold depth of water in the shallow aquifer required for “revap” or percolation to the deep aquifer 

 to occur

ROfac Percentage of the total surplus water that becomes runoff

RRAWFLOW Rainfall-Response Aquifer and Watershed Flow Model

RMSE Root mean squared error

RTI Research Triangle Institute

SAC-SMA Sacramento Soil Moisture Accounting Model

SALCC South Atlantic Landscape Conservation Cooperative

SARP Southeast Aquatic Resources Partnership

SECO Soil evaporation compensation factor

SCASC Southeast Climate Adaptation Science Center, U.S. Department of the Interior

SEEP Seepage coefficient representing infiltration losses to deep aquifer storage 

SERAP Southeast Regional Assessment Project

SEUS Southeastern United States

Sol_AWC Available water capacity of the soil layer

SRES Special Report on Emissions Scenarios

SSURGO Soil Survey Geographic database

STATSGO State Soil Geographic database

SURLAG Surface runoff lag coefficient

SWAT Soil and Water Assessment Tool

TEMP Temperature

TOPMODEL Physically based, semi-distributed topographical watershed model

USDA U.S. Department of Agriculture

USEPA U.S. Environmental Protection Agency

USFS Forest Service, U.S. Department of Agriculture

USGS U.S. Geological Survey, U.S. Department of the Interior

WaSSI Water Supply Stress Index model

WATER Water Availability Tool for Environmental Resources

WATERFALL® Watershed Flow and Allocation Modeling System

WHC Water holding capacity

Caldwell, Peter V.; Kennen, Jonathan G.; Hain, Ernie F.; Nelson, Stacy A.C.; Sun, Ge; 

McNulty, Steve G. 2020. Hydrologic modeling for flow-ecology science in the Southeastern United States 

and Puerto Rico. e-Gen. Tech. Rep. SRS-246. Asheville, NC: U.S. Department of Agriculture Forest 

Service, Southern Research Station. 77 p. 

An understanding of the applicability and utility of hydrologic models is critical to support the effective 

management of water resources throughout the Southeastern United States (SEUS) and Puerto Rico (PR). 

Hydrologic models have the capacity to provide an estimate of the quantity of available water at ungauged 

locations (i.e., areas of the country where a U.S. Geological Survey [USGS] continuous record gauge 

is not installed) and provide the baseline flow information necessary to develop the linkages between 

water availability and characteristics of streamflow that support ecological communities (i.e., support the 

development of flow-ecology response models). This report inventories and then directly examines and 

compares a subset of hydrologic models used to estimate streamflow at a number of gauged basins across 

the SEUS and PR. This effort was designed to evaluate, quantify, and compare the magnitude of error and to 

investigate the potential causes of error associated with predicted streamflows from seven hydrologic models 

of varying complexity and calibration strategy. This was accomplished by computing and then comparing 

classical hydrologic model fit statistics (e.g., mean bias, coefficient of determination [R2], root mean squared 

error [RMSE], Nash-Sutcliffe Efficiency [NSE]) and understanding the bias in the prediction in these and 

a subset of ecologically relevant flow metrics (ERFMs). Additionally, streamflow predictions from a larger 

regional-scale hydrologic model were compared to those of several fine-scale hydrologic models under a 

range of hypothetical climate change scenarios to determine the range of predicted streamflow responses to 

fixed climate perturbations. A pilot study was conducted using predicted streamflow and boosted regression 

trees to develop a set of predictive flow-ecology response models to assess the potential change in fish 

species richness in the North Carolina Piedmont under several scenarios of water availability change. 

This report is intended to provide a general assessment of all the tools and techniques available to support 

hydrologic modeling for flow-ecology science in the SEUS and PR. It is our hope that the approach used 

herein to understand differences in streamflow predictions among a subset of hydrologic models that 

have been applied in the SEUS for developing flow-ecology response models will provide water resource 

managers and stakeholders with an informed pathway for developing the capacity to link streamflow 

and ecological response and an understanding of some of the limitations associated with these type of 

modeling efforts. 

Keywords: Ecological flows, fish species richness, flow alteration, flow-ecology models, hydrologic models, 

water supply, water withdrawals.

How do you rate this publication? 

Scan this code to submit your feedback 

or go to http://www.srs.fs.usda.gov/pubeval.
In accordance with Federal civil rights law and U.S. 

Department of Agriculture (USDA) civil rights regulations 

and policies, the USDA, its Agencies, offices, and employees, 

and institutions participating in or administering USDA 

programs are prohibited from discriminating based on race, color, 

national origin, religion, sex, gender identity (including gender 

expression), sexual orientation, disability, age, marital status, family/

parental status, income derived from a public assistance program, 

political beliefs, or reprisal or retaliation for prior civil rights activity, 

in any program or activity conducted or funded by USDA (not all bases 

apply to all programs). Remedies and complaint filing deadlines vary by 

program or incident.

Persons with disabilities who require alternative means of 

communication for program information (e.g., Braille, large print, 

audiotape, American Sign Language, etc.) should contact the 

responsible Agency or USDA’s TARGET Center at (202) 720-2600 

(voice and TTY) or contact USDA through the Federal Relay Service 

at (800) 877-8339. Additionally, program information may be made 

available in languages other than English.

To file a program discrimination complaint, complete the USDA 

Program Discrimination Complaint Form, AD-3027, found online at 

http://www.ascr.usda.gov/complaint_filing_cust.html and at any USDA 

office or write a letter addressed to USDA and provide in the letter 

all of the information requested in the form. To request a copy of the 

complaint form, call (866) 632-9992. Submit your completed form or 

letter to USDA by: (1) mail: U.S. Department of Agriculture, Office of 

the Assistant Secretary for Civil Rights, 1400 Independence Avenue, 

SW, Washington, D.C. 20250-9410; (2) fax: (202) 690-7442; or (3) email: 

program.intake@usda.gov.

USDA is an equal opportunity provider, employer, and lender.