Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria, 

Volume 1 Estuaries 

 

Appendix E: Uncertainty Analysis for

Water Quality Simulation Modeling
 

November 30, 2012
 
 

Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria, 

Volume 1 Estuaries 

Contents 

Appendix E. Uncertainty Analysis for Mechanistic Modeling.......................................... E-1
 

E.1 Introduction ...................................................................................................................E-1
 

E.2 Watershed (LSPC) Model Parameters ..........................................................................E-2
 

E.2.1 Watershed Hydrology Model Parameters.........................................................E-2
 

E.2.2 Watershed Water Quality Model Parameters ...................................................E-4
 

E.3 Procedure.......................................................................................................................E-5
 

E.3.1 Parameter Uncertainty ......................................................................................E-5
 

E.3.2 Monte Carlo Sampling ......................................................................................E-6
 

E.3.3 Input to WASP7 ................................................................................................E-7
 

Figures 

Figure E-1. Example of a cumulative likelihood function that summarizes the range of
 

TN loadings produced by an advance sensitivity analysis of a LSPC
 

watershed model.......................................................................................................E-7
 

Tables 

Table E-1. LSPC watershed model parameters investigated and ranges defining the 95 

percent confidence intervals employed in advanced sensitivity analyses of the
 

Pensacola and Tampa watershed-estuary system models ........................................E-5
 

E-i 
 

 

 

 

 

Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria, 

Volume 1 Estuaries 

Appendix E. Uncertainty Analysis for Mechanistic Modeling 

E.1 Introduction 

Florida watershed estuary systems are modeled using Loading Simulation Program in C++ 

(LSPC), Environmental Fluid Dynamics Code (EFDC), and Water Quality Analysis Simulation 

Program Version 7.3 (WASP7). LSPC simulates watershed hydrology and water quality and 

estimates the water flow and nutrient loading at inlets to a coupled estuary. EFDC and WASP7 

simulate the water flow and quality, respectively, within the estuary system. Those system 

models were calibrated to effectively simulate hydrological and water quality conditions 

observed in the system. For instance, the values of hydrological and water quality parameters of 

a watershed are adjusted within reasonable bounds until the hydrological and water quality 

conditions observed at one or more USGS gauges in the watershed are replicated with reasonable 

accuracy. The resulting calibrated watershed estuary systems accurately simulate the impact of 

upstream watershed conditions on downstream estuary water quality. 

Because the calibration of the system model requires the adjustment of hydrological and water 

quality model input parameter values, it is logical to assume that there is uncertainty associated 

with the parameters. Moreover, because model parameters values are adjusted to achieve 

simulated results that reasonably replicate target conditions, it follows that simulated outputs are 

sensitive to model input values. As such, it is unreasonable to presume that the set of model input 

parameter values that produce the best calibrated model is, in fact, the true set of parameter 

values that most exactly reflects the real system being modeled. Although the set of calibrated 

model parameter values is likely close to reality, the variability of simulated model conditions 

due to variability in model input parameter values can be investigated to demonstrate the degree 

to which simulated downstream estuarine water quality conditions can change with respect to 

changes in hydrological and water quality parameter values within realistic ranges. Such an 

analysis is herein termed an advanced sensitivity analysis. 

Whereas traditional model sensitivity analyses perturb a subset of model input parameters within 

a small range of values to gauge the robustness of the calibrated model, an advanced sensitivity 

analysis adjusts model input parameters within a larger range that reflects the values that model 

parameters can take in nature. Subsequently, simulated results include a range in model 

outcomes that can plausibly occur. 

In an advanced sensitivity analysis, the process of adjusting model parameters and assessing 

resulting model outcomes is guided by Monte Carlo sampling, wherein a single value is defined, 

or sampled, for each queried model input parameter, and the resulting model is executed; 

simulation outcomes are then tabulated and assessed. That process is repeated numerous times 

until the suite of values defined (sampled) for each input model parameter sufficiently reflects 

the prescribed range of realistic values. The numerous sets of simulation outcomes are 

aggregated and reduced to a range of outcomes (lower and upper bounds). Such an analysis 

applied to an LSPC model, for instance, could produce a range of nutrient loads at estuary inlets. 

Although ranges of model results can be informative for describing the worst and best case 

scenarios, it is often desirable to understand the likelihood of those scenarios; perhaps the lower 

and upper bounds on a model outcome are highly improbable, whereas model outcomes between 

E-1 
 

 

 

Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria, 

Volume 1 Estuaries 

those bounds are more likely to occur. Thus, an element of probability must be introduced into 

the advanced sensitivity analysis. The calibrated model introduces such probability where one 

assumes the calibrated value for a input model parameter is the mean, or expected value for that 

parameter in the system being modeled. By definition, the calibrated model outcomes reflect the 

expected simulated conditions. Variability is introduced by prescribing 95 percent confidence 

intervals for key input model parameters; the values for the 95 percent confidence intervals are 

generally guided by published parameter ranges, modeler insight, and field data. 

The purpose of applying an advanced sensitivity analysis to Florida watershed-estuary systems is 

to characterize the impact of varying watershed characteristics on corresponding downstream 

estuary water quality and ultimately assess how much estuary water quality can vary in reality. 

Specifically, watershed hydrology and water quality inputs are adjusted within prescribed ranges, 

the range of simulated biochemical oxygen demand (BOD), TN, and TP loadings to the 

corresponding estuary are probabilistically assessed, and various percentiles (e.g., 5th percentile, 

95th percentile) of simulated constituent loadings are input into the WASP estuary water quality 

model, and subsequent estimates estuary endpoint are calculated. The advanced sensitivity 

analysis was applied to the Pensacola and Tampa watershed-estuary systems. Those two systems 

represent relative extremes in watershed land use: mostly forested and mostly urban, 

respectively. By applying the advanced sensitivity analysis to those two systems, one can 

reasonably assess a full range of the potential variability in predictions of estuarine water quality 

across all modeled Florida estuaries. 

E.2 Watershed (LSPC) Model Parameters 

As previously mentioned, watershed model parameter values are varied to produce variability in 

model simulated constituent loadings to the linked estuary. Modelers identified a subset of 

watershed model parameters to which simulated constituent loadings are most sensitive. The 

parameters fall into two categories: hydrology and water quality. The parameters adjusted in the 

advanced sensitivity analyses are briefly described below. 

E.2.1 Watershed Hydrology Model Parameters 

The following parameters characterize the hydrological aspects of the modeled watershed, and, 

as such, they predominantly affect the simulated watershed hydrology; it follows logically that 

simulated water quality conditions are sensitive to these parameters, as well. 

•	 Lower zone nominal soil moisture storage (LZSN), inches, is related to both 

precipitation patterns and soil characteristics.
 

•	 Index to mean soil infiltration rate (INFILT), inches/hour, controls the partitioning of 

available moisture from precipitation into surface flow, subsurface flow, and storage 

components and is a function of soil characteristics and land use. 

•	 Variable groundwater recession rate parameter (KVARY), inches-1, is employed to 

characterize the groundwater recession rate that varies with season and groundwater 

levels. 

•	 Base groundwater recession rate (AGWRC), days-1, is the ratio of current groundwater 

discharge to that 24 hours prior. 

E-2 
 

 

 

 

 

Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria, 

Volume 1 Estuaries 

•	 Deep infiltration fraction (DEEPFR), unitless, is the fraction of infiltration that is 

apportioned to deep aquifers; {1 – DEEPFR} is infiltration retained in active 

groundwater storage and is related to geology and groundwater recharge. 

•	 Streambed-based riparian vegetation evapotranspiration (ET) fraction (BASETP), 

unitless, is the fraction of potential ET that stream-adjacent vegetation uptakes. 

•	 Model land ET fraction (AGWETP), unitless, is the fraction of modeled land that is 

susceptible to groundwater storage-based ET because of a shallow groundwater table 

(e.g., fraction of model covered by wetlands). 

•	 Vegetation-retained rainfall fraction (CEPSC), unitless, is the percentage of rainfall 

that never reaches land surface and eventually evaporates (monthly varying parameter), 

and is related to type of vegetation present on a parcel of land; all monthly values of 

CEPSCs for each land use and soil class are adjusted by a single factor. 

•	 Nominal upper-zone soil moisture storage (UZSN), inches, dictates the amount of 

water stored in the upper zone and made available for ET (monthly varying parameter); 

related to surface soil conditions and land use; all monthly values of UZSN for each land 

use and soil class are adjusted by a single factor. 

•	 Interflow-from-surface coefficient (INTFW), unitless, determines the amount of water 

that enters the subsurface from surface detention storage and becomes interflow (monthly 

varying parameter); related to soils, topography, and land use. All monthly values of 

INTFW for each land use and soil class are adjusted by a single factor. 

•	 Interflow recession coefficient (IRC), unitless, is the ratio of current daily interflow 

discharge to that of the previous day (monthly varying parameter); like INTFW, it is 

related to soils topography and land use. All monthly values of IRC for each land use and 

soil class are adjusted by a single factor. 

•	 Index to lower zone ET (LZETP), unitless, is a coefficient that governs ET potential 

(monthly varying parameter) and is related to vegetation type and root depth; to preserve 

the calibrated temporal variation, all monthly values of LZETP for each land use and soil 

class are adjusted by a single factor. 

Each modeled watershed has a number of prescribed land uses (e.g., urban pervious, agricultural) 

and soil classes (e.g., sandy, clayey). The Pensacola watershed model has 15 land uses and 3 soil 

classes; the Tampa watershed model has 16 land uses and 5 soil classes. Each of those 

parameters can have a unique value for each land use and soil class, although certain land uses 

behave similarly and their parameters for those tied land uses take the same value. On the basis 

of those considerations, 300 hydrological parameters were adjusted in the advanced sensitivity 

analysis for the Pensacola watershed model and 525 hydrological parameters were adjusted in 

the advanced sensitivity analysis for the Tampa watershed model. 

E-3 
 

 

 

Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria, 

Volume 1 Estuaries 

E.2.2 Watershed Water Quality Model Parameters 

The following parameters describe the water quality characteristics of the modeled watershed. 

Variation in the values of these parameters produces variability in simulated watershed water 

quality (BOD, TN, and TP loadings), although simulated watershed hydrology is not sensitive to 

these parameters. 

•	 Rate of surface runoff to remove 90 percent of stored constituent (WSQOP), 

inches/hour, is the amount of runoff necessary to remove 90 percent of the accumulated 

constituent on land surface. 

•	 Constituent accumulation rate (ACCUM), pounds/acre/day, is the rate at which a 

constituent builds up on the ground surface in a watershed; varying monthly, the temporal 

variation in this parameter is more important than the magnitude of the individual 

monthly values. As such, all monthly values of ACCUM for each land use, soil class, and 

constituent are adjusted by a single factor. 

•	 Storage limit of constituent (SQOLIM), pounds/acre, is the maximum constituent mass 

that can be stored on the ground surface in a watershed; although this parameter varies 

monthly, in modeling the monthly varying values are a multiple of the ACCUM value 

assigned to the corresponding constituent, land use, and soil class. 

•	 Interflow constituent concentration (INTERCONC), mg/L, is the monthly varying 

concentration of a constituent in the upper part of the unsaturated zone; varying monthly, 

the temporal variation in this parameter is more important than the magnitude of the 

individual monthly values. As such, all monthly values of INTERCONC for each land 

use, soil class, and constituent are adjusted by a single factor; that single factor is queried 

in the advanced sensitivity analysis. 

•	 Groundwater constituent concentration (GRNDCONC), mg/L, is the monthly varying 

concentration of a constituent in the subsurface saturated zone; varying monthly, the 

temporal variation in this parameter is more important than the magnitude of the 

individual monthly values. As such, all monthly values of GRNDCONC for each land 

use, soil class, and constituent are adjusted by a single factor. This parameter must be less 

than or equal to the INTERCONC value assigned to the corresponding constituent, land 

use, and soil class. 

•	 Constituent first-order decay rate (DECAY), day-1, is the rate at which the in-stream 

concentration of a constituent is chemically mitigated because of the transformation of 

the constituent into a product; this parameter varies by stream type. 

•	 Temperature correction for first-order decay (TCDECAY), unitless, a temperaturebased adjustment that affects the in-stream decay rate of a constituent; this parameter 

varies by stream type. 

•	 BOD decay rate at 20 °C (KBOD20), hour-1, is the rate at which BOD decays at
 

standard temperature; this parameter varies by stream type.
 

•	 BOD settling rate (KBODSET), meters/hour, the removal rate of in-stream BOD due to 

settling; this parameter varies by stream type. 

E-4 
Table E-1. LSPC watershed model parameters investigated and ranges defining 

the 95 percent confidence intervals employed in advanced sensitivity analyses 

of the Pensacola and Tampa watershed-estuary system models 

Parameter Pensacola Watershed Tampa Watershed 

INFILT [0.05, 0.5] ±50% 

AGWRC [0.950, 0.999] [0.850, 0.999] 

BASETP [0, 0.1] [0, 0.2] 

AGWETP (only wetlands) [0.2, 0.8] [0.5, 0.9] 

CEPSC ±50% ±50% 

INTFW ±50% ±100% 

LZSN [6, 15] [6, 15] 

KVARY [0, 1] ±50% 

DEEPFR [0, 0.2] [0, 0.6] 

UZSN [0, 2] ±100% 

Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria, 

Volume 1 Estuaries 

As with the hydrological parameters defined above, the water quality parameters can vary with 

land use and soil class. Additionally, the parameters can vary by constituent, and some vary by 

stream type. On the basis of those considerations, in addition to the fact that some land uses and 

constituents are modeled to behave similarly with respect to some of the water quality 

parameters, 1,387 water quality parameters were investigated in the advanced sensitivity analysis 

of the Pensacola watershed-estuary system; 1,870 water quality parameters were investigated in 

the advanced sensitivity analysis of the Tampa watershed-estuary system. 

E.3 Procedure 

E.3.1 Parameter Uncertainty 

As stated previously, the desired outcome of the advanced sensitivity is to understand how 

estuary water quality varies because of variability in upstream watershed hydrology and water 

quality. More pointedly, it is desirable that the likelihood associated with predicted estuary water 

quality be characterized. To characterize model outcomes probabilistically, uncertainty in 

watershed model inputs must be defined. Such uncertainty is warranted by the fact that whereas 

the set of calibrated watershed model parameters provides the best solution, the set of parameter 

values might not exactly reflect watershed conditions in reality. However, it is assumed that the 

calibrated parameter values are the expected, or mean, parameter values. 

In addition to identifying the parameters that require attention in the advanced sensitivity 

analysis, modelers prescribed ranges of values within which the parameters could plausibly vary. 

For a model parameter, that range of values is informed by modeling insight, a knowledge of the 

modeled watershed, and published data. For the purposes of Monte Carlo analysis, those ranges 

of values were interpreted as the 95 percent confidence interval. In conjunction with the 

assumption that a parameter’s calibrated value is the expected (e.g., mean value) and assumption 

of normality in the stochastic distribution of watershed model parameters, the prescribed 95 

percent confidence intervals characterize the likelihoods associated with each model parameter 

listed above. The 95 percent confidence intervals prescribed for the Pensacola and Tampa 

watershed models are listed in Table E-1. 

E-5 
Parameter Pensacola Watershed Tampa Watershed 

IRC [0.50, 0.85] [0.5, 0.85] 

LZETP ±50% ±50% 

WSQOP [0.1, 0.9] [0.1, 0.9] 

ACCUM ±100% ±100% 

SQOLIM [1, 6] [1, 6] 

INTERCONC ±100% ±100% 

GRNDCONC ±100% ±100% 

DECAY (only TN, TP) TN: [0.00, 0. 25] TN: [0.00, 0. 25] 

TP: [0.000, 0.025] TP: [0.000, 0.025] 

TCDECAY (only TN, TP) [1.00, 1.07] [1.00, 1.07] 

KBOD20 [0, 0.2] [0, 0.2] 

KBODSET [0, 0.1] [0, 0.1] 

 

 

Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria, 

Volume 1 Estuaries 

E.3.2 Monte Carlo Sampling 

To convey the prescribed uncertainty in model input parameters to model outputs, a Monte Carlo 

sampling approach was employed. Monte Carlo sampling execution begins with selecting 

(sampling) of a single value for each uncertain model parameter between the modeler-defined 

lower and upper bounds. The collection of selected values comprises a complete set of model 

inputs necessary to execute a single instance of the watershed model; that set of sampled values 

is alternatively called a realization. The model is subsequently executed and the resulting model 

outcomes are tabulated. In this application of Monte Carlo analysis, the model outcomes of 

interest are the simulated annual BOD, TN, and TP loadings at subwatersheds that are inlets to 

the coupled estuary system. The process of sampling a realization from the set of stochastic 

parameters, executing the resulting model, and tabulating the model outcomes is repeated many 

times until the set of sampled realizations sufficiently reflects the prescribed uncertainty in 

model input parameters. The sampling of a parameter is guided by the probability distribution 

defined for that parameter; values associated with higher probabilities are sampled more 

frequently than those associated with lower probabilities. 

After Monte Carlo sampling is complete, the many instances in key model predictions (BOD, 

TN, TP loadings at estuary inlets) are aggregated probabilistically such that a likelihood curve 

can be constructed to represent both the range in plausible model outcomes and the likelihood 

associated with the outcomes within that range. Figure E-1 is an example of such a cumulative 

likelihood curve reflecting the variability in TN loadings produced by an advanced sensitivity 

analysis of a non-Floridian watershed. For all values in the range of simulated TN loadings, the 

cumulative likelihood curve defines the probability of occurrence, solely on the basis of 

variability in model input parameter values. For instance, the curve suggests that it is 95 percent 

likely that the predicted spring TN loading will be less than 7,000 lb and 5 percent likely that the 

predicted spring TN loading will be less than 5,500 lb. The median predicted TN loading is 

approximately 6,200 lb. Whereas traditional modeling would produce a single predicted spring 

TN loading, the advanced sensitivity analysis provides a range of values and likelihoods 

associated with predictions within that range. 

E-6 
 

 

 

 

 

Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria, 

Volume 1 Estuaries 

0 

10 

20 

30 

40 

50 

60 

70 

80 

90 

100 

0 5000 10000 15000 20000 

Cumulative Likelihood 

Total Nitrogen Spring Loading (lb) 

Total Nitrogen Spring Loading Likelihood Function 

Figure E-1. Example of a cumulative likelihood function that summarizes the range of TN loadings produced by 

an advance sensitivity analysis of a LSPC watershed model 

E.3.3 Input to WASP7 

The results of the Monte Carlo sampling are cumulative likelihood curves, similar in structure to 

that presented in Figure E-1. The curves define the range of plausible predicted watershed BOD, 

TN, and TP loadings to the coupled estuary, as well as the likelihood associated with the 

predicted loadings from each watershed subbasin that is an inlet to the coupled estuary. 

To further simulate the range in plausible estuarine water quality predictions, the probabilistic 

information must be input to the WASP7 model of the coupled estuary. The multiday run times 

associated with the WASP7 models for the Pensacola and Tampa watersheds preclude a 

complete Monte Carlo analysis, wherein the uncertain constituent loadings are exhaustively 

sampled and resulting realizations inform numerous WASP7 model executions. 

To expedite the analysis and simplify the desired predictions, the 5th and 95th percentile loadings 

predicted for the estuary inlets were gleaned from the suite of cumulative likelihood curves. 

Intuitively, the 5th percentile loadings reflect plausibly low constituent source conditions in the 

estuary; conversely, the 95th percentile loadings reflect plausibly high constituent source 

conditions in the estuary. The calibrated WASP7 estuary model was executed for both 

conditions. The resulting set of WASP7 predictions reflect a plausible range in estuarine water 

quality with respect to BOD, TN, and TP concentrations. 

E-7