Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries



# Appendix B: Statistical (Stressor

Response) Analys is

November 30 , 2012Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries



### Contents

Appendix B. Empirical Analyses

.......................................................................................... B-1

B.1 Data

............................................................................................................................... B-1

B.2 Derivation of Chl- a Criteria to Meet Water Clarity Targets

......................................... B-1

B.2.1 Data Screening

.................................................................................................. B-1

B.2.2 Statistical Model

............................................................................................... B-4

B.3 Derivation of Numeric Nutrient Criteria to Meet Algal Bloom Frequency Endpoint

...................................................................................................................... B-7

B.4 Derivation of TN and TP Criteria to Achieve Chl- a Criteria

........................................ B-8

B.4.1 Data Screening

.................................................................................................. B-8

B.4.2 Statistical Model

............................................................................................... B-9

B.5 Computing Upper and Lower Bounds of Available Data

........................................... B-13

B.6 Nutrient Targets for Downstream Protective Value Calculation

................................ B-13

B.7 References

................................................................................................................... B-15

B

- iTechnical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries



### Figures

Figure B -1. Example of combining samples from neighboring stations in Indian River Lagoon. Filled gray circles show two sampling locations for which data were combined. Open circles show other sampling locations. .........................................B-2

Figure B -2. Example of estimating depth at a sampling location from sounding data. Left panel: open grey circles, sounding locations; contours indicate fitted bathymetric surface; filled red circle, sampling locati on. Right panel: depth predicted by nonparametric thin plate spline plotted against observed sounding depth. Filled red circle indicates the depth estimated for the sampled location. ......................................................................................................B-3

Figure B -3. Example of comparing measured Secchi Depth with estimated depth at the sampling location (shown as the horizontal line). Estimated depth at sampling location computed from nonparametric model prediction and mean tidal range, as described in the text. .........................................................................B-3

Figure B -4. Example of estimated relationship between chl- a and K d . Red horizontal line

shows K d corresponding with seagrass depth target for the segment. Red vertical arrow shows annual geometric mean chl -a concentration predicted to

be associated with K d target. Green line segment shows the 5 th to 95 th
percentile range of observed annual geometric mean values for chl

- a. Open circles show measured annual average values of K d (adjusted for the effect of turbidity and color) and chl- a. ..................................................................................B-5

Figure B -5. Estimates of the slopes between K d and chl-a in different estuarine segments. Two panels from top to bottom show Northwest and Eastern estuaries.

Vertical lines show estimates of the 9 0% confidence intervals on each slope, open circles show mean estimate of slope. ...............................................................B-6

Figure B -6. Example of model of the relation ship between annual geometric mean chl

- a concentration and bloom frequency. Open circles: observed annual frequency

of chl- a concentrations exceeding 20 μg/L versus annual geometric mean chl

a concentration for the same year; solid black line: modeled relationship

between mean chl -a concentration and bloom frequency; horizontal red line: targeted bloom frequency of 10%; vertical red arrow: annual geometry mean

chl

- a concentration associated with targeted bloom frequency; green line segment: 5 th to 95 th perc entile range of observed annual geometric mean chl- a concentrations. ..........................................................................................................B-8

Figure B

- 7. Example of estimated stressor

- response relat ionship between TN and chl- a. Blue line: segment mean relationship. Grey lines: relationships estimated

within different stations. Red horizontal line shows the chl- a criterion associated with the water clarity endpoint for each segment. Green segment

show s the range of observed annual geometric mean TN values. Open circles show observed values of annual geometric mean chl- a and TN. .............................B-9

B

- iiTechnical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries

Figure B -8. Estimates of the slope of the linear relationship between annual geometric

mean TP and chl -a among different segments. Two panels from top to bottom show Northwest and Eastern estuaries. Open circles: mean estimate of slope;

vertical lines: estimated 90% confidence intervals. ...............................................B-11

Figure B -9. Estimates of the slope of the linear relationship between annual geometric

mean TN and chl -a among different segments. Two panels from top to bottom show Northwest and Eastern estuaries. Open circles: mean estimate of slope; vertical lines: estimated 90% confidence intervals. ................................B-12

Figure B -10. Map of station locations in St. Lucie estuary. Stations labeled in order of maximum observed salinity (highest salinity is numbered 1). Station circled

in red is loc ated in the estuary pass. .......................................................................B-14

Figure B -11. Observed salinity and nutrient concentrations at estuary pass of St. Lucie. ........B-14

Figure B

- 12. Illustration of calculation of DPVs using a mixing/dilution relationship for Indian River Lagoon. Black diamond shows the estimated conditions in seawater. Green circles show proposed TP criteria for different segments. Green triangles show DPVs computed for each segment. Open circles show long- term average salinity and TP and different stations in the estuary. ...............B-15

B

- iiiTechnical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries



### Appendix B. **Empirical Analyses**

Analyses of data collected from the field inform criteria derivation by identifying nutrient concentrations that are protective of biological endpoints.



### B.1 **Data**

Water quality data were retrieved from the Impaired Waters Rule ( IWR ) Run 40 database (FDEP 2010) and screened based on whether they were located within pre-defined estuary segments, yielding a total of 1,126,560 water quality samples collected at 26,495 sites. Different screens (described below) were applied to th ese data depending on the requirements of different analyses, which reduced the size of the database.

Measurements of TN, TP, chlorophyll a ( chl -a), nitrate+nitrite ( NO 3 + NO 2 ), and total Kjeldahl nitrogen were all log -transformed to reduce the skewness of the distributions.

Measurements of Secchi depth were converted to an estimate of the light attenuation coefficient (K d ) using the following relationship (Holmes 1970):

1. 44 k d = Secchi Dept h



### B.2 **Derivation of Chl -a**  **Criteria to Meet Water Clarity Targets**

**B.2.1 Data**  **S creening** 

To derive chl- a criteria to meet water clarity targets, data retrieved from IWR Run 40 were

further screened to include only samples with con current measurements of Secchi depth and chl

a. Chl- a measurements that were recorded as being below any detection lim it that was 5 μg/L or greater were also excluded.

In each estuary, all sampling locations were projected into the Universal Transverse Mercator coordinate system ( UTM Zone 17). Then, data collected at different times from sites within 350 m of one another were combined and analyzed as if collected from the same location ( Figure

B-1 ) . The allowed separation distance of 350 m was selected by examining neighboring sites in which the data record at one site began after the data record at another site ended. In these pairs

of site s, the slight change in sampling location may have resulted from a change in the datum on which the geographic location was based or a change in sampling protocol. Furthermore, EPA

determined that natural characteristics of sites (e.g., salinity) within 350 m of one another in an

estuary were assumed to be similar, and hence, relationships between Secchi depth and chl- a at these closely located sites were modeled as if the observations were collected at the same site.

B

- 1Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries

Figure B-1. Example of combining samples from neighboring stations in Indian River Lagoon. Filled gray circles
show two sampling locations for which data were combined. Open circles show other sampling locations.

Depth at s ample d locatio ns was estimated using digital depth sounding data acquired from

NOAA’s National Geophysical Data Center ( http://www.ngdc.noaa.gov/mgg/bathymetry/relief.html ). Depth sounding data in a 1 km × 1 km box surrounding each sampled location was selected, and a smooth bathymetric surface was estimated using a nonparametric thin plate spline with a maximum of 30 degrees of freedom (Wood and Augustin 2002). The maximum number of degrees of freedom for the nonparame tric fit sets the upper limit on the degree to which the fitted surface “wiggles ,” and exploratory analysis indicated that 30 degrees of freedom was sufficient to represent variations in depth, without overfitting the data (Wood and Augustin 2002) . Once the nonparametric surface was fit, the depth at the sampled location was estim ated from the mean depth predicted by the nonparametric model. To account for errors in the predicted depth, an estimate of the upper 90 th
prediction interval for the fitted value (i.e., mean plus 1.64 times the standard deviation of the

residual deviation ) was used as the estimated depth.

Data collected from sample locations that were greater than 150 m from a recorded depth sounding were excluded from the da taset. B ecause depth can vary greatly over short distances and because depth soundings were relatively close to one another ( Figure B-2 ), maintaining a relatively low cut -off distance of 150 m between sample location and the sounding location was

reasonable.

B

- 2Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries

Figure B-2 . Example of estimating depth at a sampling location from sounding data. Left panel: open grey circles,
sounding locations; contours indicate fitted bathymetric surface; filled red circle, sampling location. Right panel:
depth predicted by nonparametric thin plate spline plotted against observed sounding depth. Filled red circle
indicates the depth estimated for the sampled location.

Each Secchi depth measurement was compared to the estimated depth at the sample location ( Figure B-3). Since the depth sounding is referenced to mean lower -low water, the mean tidal range estimated for that location was added to the sounding depth before comparing with Secchi depth. Tidal ra nge data for different estuaries w ere obtained from NOAA’s Center for Operational Oceanographic Products and Services ( http://tidesandcurrents.noaa.gov ) and the maximum tidal range observed in each segment w as used . In some segments, tidal range information was not available, so tidal range data from nearby stations were used.

Figure B-3 . Example of comparing measured Secchi Depth with estimated depth at the sampling location (shown
as the horizontal line). Estimated depth at sampling location computed from nonparametric model prediction and
mean tidal range, as described in the text.

B

- 3Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries

Secchi depth measurements that were greater than the estimated depth of the estuary at the sample location were treated as censored measurements, known only to be greater than the estimated depth. L ight attenuation coefficients that were estimated from these censored S ecchi depth measurements were also treated as censored data. Sampling locations at which the majorit y of Secchi depth measurements (i.e., greater than 50%) were censored were excluded from the

calculation.

Light attenuation coefficients, chl -a, turbidity, and color at each sampling location were summarized as annual geometric means. Sampled values recorded as being less than the detection limit were included in the computation of annual geometric means as half of the

detection limit. At le ast four different samples of each parameter (Secchi depth, chl -a, turbidity, and color), including at least one sample collected in the summer (May –September) and one sample col lected in the winter (October –April), were required to compute annual geometric means for that parameter. The s e requirements ensured that annual geometric mean values were estimated with sufficient precision for subsequent analysis. Relationships were estimated for an estuary segment only if at least t wo distinct stations with data satisfying all pre vious screens were available in the segment, to ensure the data sufficiently represent spatial variations within each segment.

**B.2.2 Statistical**  **M odel** 
Relationships between annual geometric mean water clarity, as quantified by light attenuation coefficient, and chl

- a were estimated using mixed effects models that represented the hierarchical structure of the data within sampling locations and within segments (Gelman and Hill 2007). In addition to chl-a, the model also quantified the effects of turbidity and color on observed water clarity. A model of the following form was fit:

log ( k ഥ d

, ijk ) = b 0, jk + b 1, jk log ൫ chl ijk ത ത ത ത ത ത ത ത ത ത ൯ + b 3, jk log ൫ color ijk ത ത ത ത ത ത ത ത ൯ + b 2, jk log ൫ turb ijk ത ത ത ത ത ത ത ത ത ത ത ൯ + ε i

w here k ഥ d

, ijk , ത chl ത ത ത ത ijk , turb ijk ത ത ത ത ത ത ത ത ത ത , and color ത ത ത ത ത ത ത ത ijk are the annual geometric mean light attenuation
coefficient, chl -a, turbidity, and color for year i at station k in segment j

, b 0..3, jk are regr ession

coefficients for station k and segment j , and ε i is a normally distribu ted error. The hierarchical

structure of the data was represented by the regression coefficients, which we re each compute d as the sum of an overall mean value of the coefficient, a segment

- specific value, and a station

specific value:

b

m
, jk = b

m + δ

m
, j + ∆
m
, k

w here b

m is the overall mean value for coefficient m , δ

m
, j is the random effect for segment j , and

∆
m
, k is the random effect for station k . Random effects associated with segment s and station s

we re assumed to be normally distributed with a mean value of zero.

In the northwest region of Florida (Panhandle and Big Bend), turbidity and color measurements were not collected in enough samples to include in the model, so in this region only the effect of

chl -a on K d was modeled:

log ( k ഥ ത ത ത ത ത ത ത ത ൯ d

, ijk ) = b 0, jk + b 1, jk log ൫ chl ijk

B

- 4Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries

Models were fit using the package lme4 in R (R Development Core Team 2009).

Chl- a values associated with depth targets ( D t arget ) for each segment we re computed by first converting the D target for each segment to a K d target using the following relationship, which computes the light attenuation coefficient that is necessary to achieve 20% light attenuation at the depth target:

− ln (0.2)

= k d ( target ) D target

Then, the segment- specific relationship between K d and chl- a was extracted from the fitted model, assuming that turbidity and color were constant at the mean value for the segment. This relationship was used to compute the chl- a criterion as the chl

- a concentration that was associated with the targeted K d value ( Figure B-4 ).

Figure B-4

. Example of estimated r elationship between chl

- a and K d . Red horizontal line shows K d corresponding
with seagrass depth target for the segment

. Red vertical arrow show s annual geometric mean chl

- a concentration predicted to be associated with K d target. Green line segment shows the 5 th to 95 th percentile range of observed

annual geometric mean values for chl -a. Open circles show measured annual average values of K d (adjusted for the
effect of turbidity and color) and chl

- a.

In some c ases, the chl

- a concentration associated with the K d target was less than the minimum annual geometric mean chl -a concentration observed in the segment. In th ese case s , the chl

- a criterion value was set at the lower boun d of the observed range of data. Conversely, in some segments, the chl

- a concentration associated with the K d target was greater than the maximum geometric mean chl

- a concent ration observed in the segment. In these case s, the chl- a criterion was set at the upper bound of the observed ra nge of data (see Section B.5 ). Relationships

between K d and chl

- a varied among estuaries and estuary segments ( Figure B-5 ).

Relationships with negative and near- zero slopes generally occurred in estuarine segments near the inflows from large tributaries (e.g., Segment 1403, the Upper St. Lucie River); these relationships were li kely affected by high concentrations of suspended solids and colored dissolved organic matter. Variations in the slope of the relationship between K d and chl- a were

B

- 5Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries

also expected because the relationships can depend on existing conditions in the water body . For example, increasing chl

- a concentration by 1 μg/L in waters that already have high

concentrations of chl- a would have a smaller influence on water clarity than increasing chl

- a by

that same amount in waters that have low chl-a concentrations. The hie rarchical structure of the model minimizes the potential effects of these issues by modeling changes within each sampling location, and thus, constraining the model to represent the relatively limited variations that occur specifically at each location.

Figure B-5

. Estimates of the slopes between K d and chl

- a in different estuarine segments.

T wo panels from top to bottom show Northwest and Eastern estuaries. Vertica l lines show

estimates of the 90% con fidence intervals on each slope , open circles show mean estimate
of slope.

In segments in which the slope of the relationship between K d and chl

- a was very near zero (such that the confidence intervals included zero) or negative, water clarity w as noted a s being insensitive to changes in chl-a, and no chl- a criterion value associated with the water clarity endpoint was computed. Note, though, that additional data collection would improve the precision with which these relationships were estimated.

B

- 6Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries



### B.3 **Derivation of Numeric Nutrient Criteria to Meet Algal Bloom Frequency Endpoint** 

Relationships between annual geometric mean chl -a concentrations and the frequency of phytoplankton blooms were used to estimate annual geometric mean chl- a concentrations

assoc iated with meeting the phytoplankton bloom endpoint. First, annual geometric mean chl-a

concentrations were estimated for each station . S tation

- year means were estimated from at least four distinct samples

, with at least one sample collected in the winter and one sample collected in the summer. Stations were only retained if at least 3 years of data were available, and relationships were only estimated for estuary segments in which at least 2 stations satisfied all previous data screens. The additional scre en of requiring 3 years of data for any single station (compared to only 1 year of data for K d - chl

- a models) was im posed for these models because estimated relationships were expected to vary more greatly among stations. More specifically, since these models only include a single explanatory variable, it was expected that variations in

c ovariates at other s tations would influence the results. In contrast, all of the primary explanatory

factors were included in the K d - chl -a models, lessening the potential variability of relationships among stations. Hence, a single pair of annual geometric mean K d and chl-a values would inform

model estimates.

The final set of data was modeled using mixed models by fitting a model of the following form:

p ijk ln ቆ ቇ

= d 0

, j + d 1

, j log ( chl ത ത ത ത ijk ) 1

− p ijk

w here p ijk is the probability of chl- a concentration exceeding 20 μg/L in year i

, segment j , and

station k , and chl ത ത ത ത ijk is the annual geometric mean chl

- a concentration in year i

, segment j , and

station k

. The regression coefficients include an effect that varies across segments as well as an overall mean effect as follows,

d
m

, j = d
m + δ
m

, j + ∆
m

, k

Where m takes the value of 0 or 1, d

m is the overall mean value for the regression coefficient and

δ

m

, j is a normally distributed random effect that differs for each segment, and ∆
m
, k is a normally

distributed random effect that differs for each station.

To derive criteria associated with the phytoplankton biomass endpoint, the annual geometric

chl

- a concentrations associated with p ij = 0.1 (i.e., a bloom frequency of 10%) were computed ( Figure B-6).

As with other criteria derived using stre ssor -response relationships, in some cases the criterion associated with a bloom frequency of 10% was greater than the upper bound of the range of observed geometric mean chl -a concentrations. In these cases, to avoid extrapolating models beyond the range of data, criteria were set at the upper bound of the observed chl- a concentrations.

B

- 7Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries

Figure B-6 . Example of model of the relationship between annual geometric mean chl

- a concentration and
bloom frequency. Op en circles: observed annual frequency of chl

- a concentrations exceeding 20 μg/L versus
annual geometric mean chl

- a concentration for the same year; solid black line: modeled relationship between

mean chl

- a concentration and bloom frequency; horizontal red line: targeted bloom frequency of 10%; vertical red

arrow: annual geometry mean chl

- a concentration associated with targeted bloom frequency; green line segment:

5 th to 95 th percentile range of observed annual geometric mean chl

- a concentrations.



### B.4 **Derivatio n of TN and TP Criteria to Achieve Chl**  **-a Criteria** 

Once chl -a criteria were derived to sati sfy either the water clarity or the phytoplankton bloom endpoint, nutrient-chlorophyll relationships were required to calculate TN and TP criteria that

were consistent with the chl -a criterion.

**B.4.1 Data Screening** 

To estimate relation ships between TN, TP, and chl- a, data retrieved from IWR Run 40 were further screened to include only samples with a measurement of chl

- a, and at least one

measurement of either TN, TP , or nitrate+nitrite

. Station locations were then analyzed and stations that were within 350 m of one another were combined (see Section B.2.1 ). Samples with

color measurement exceed ing 50 PCU were excluded to minimize the effects of water color on estimated relationships. Annual geometric mean values for TN, TP, nitrate+nitrite , and chl- a were computed for each station -year combination in which at least 4 samples were available in that year and at least 1 sample was collected in the summer (May

– September) and one sample

was collected in the winter (October– April). Stations were only used in the analysis if at least 3 annual means were available, and segments were only used in the analysis if at least 2 stations satisfying the screens described above were available. Here, as with the bloom frequency models, variations of the relationships among stations were expected to be greater than obs erved

for the K d - chl -a relationships, so more data w ere required for any single station to be included in

the model.

B

- 8Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries

**B.4.2 Statistical Model** 

M ixed effects models were used to represent the hierarchical structure of the data within sampling locations and segments. More specifically, in each e stuary segment, the model estimated the linear relationship between annual geometric mean chl

- a and annual mean geometric TN or TP within stations and within segments. Annual geometric mean values of TN, TP, and chl- a were first calculated from sample data for each station. Then, a model of the following form was used to model the relationships between TP and chl- a:

log ൫chl a c 0

, jk + c 1

, jk log ( TP ത ത ത ത

ijk ) ത ത ത ത ത ത ത ijk ൯ =

w here chl a ത ത ത ത ത ത ത ijk and TP ത ത ത ത

ijk are the geometric me an chl -a and TP concentrations for year i , in

station k , in segment j . The regression coefficients we re computed as the sum of an overall mean coefficient value, a segment

- specific value, and a station

- specific value as follows:

c m

, jk = c m + δ
m

, j + ∆
m

, k

w here m is equal to 0 or 1, δ
m

, j is the random effect attributed to segment j , and ∆
m

, k is the

random effect attributed to station k . The distributions of random effects at the station and segment level were assumed to be normal with a mean value of zero. A n identical model w as used to estimate the rel ationships between TN and chl- a, and in estuaries in which sufficient data

were availa ble, the same model form was used to estimate relationships between nitrate+nitrite

and chl- a. An example stressor -response model for Segment 1504 between chl- a and TN is

shown in Figure B-7.

TN and TP criteria were determined as the point at which predicted annual geometric mean chl

- a concentration was equivalent to the chl- a criterion. N itrate+nitrite target values were also estimated where data were available, to he lp interpret relationships observed between TN and

chl

- a.

Figure B-7

. Example of estimated s tressor -r esponse relationship between TN and chl

- a. Blue line: segment mean
relationship. Grey lines: relationship s estimated within different stations. Red horizontal line shows the chl

- a criterion associated with the water clarity endpoint for each segment. Green segment shows the range of
observed annual geometric mean T N values. Open circles show observed values of annual geometric mean chl

- a
and TN.

B

- 9Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries

As was observed when deriving chl- a criteria to meet water clarity endpoints, in some segments,

the candidate T N or T P criterion estimated from the stressor -response relationship was outside the range of observed annual geometric mean T N or T P concentrations. When the candidate

criteri on was greater than the upper bound of the range of value s observed for TN or TP, the

criterion was set at the upper bound. Conversely, when the candida te criterion was less than the lower bound of the range of values observed for TN or TP, the criterion value was set to the

lower bound.

Estimates of the slope of the relationship between TP and chl-a were strongly positive for the vast majority of estuary segments ( Figure B-8). The only segments in which near zero or negative slopes were observed were 1801 and 1802 (Lower St. Johns River and Trout River) and

several se gments in Charlotte Harbor. In contrast, a number of segments exhibited near zero (such that confidence intervals included zero) or negative slopes of the relationship between TN

and chl- a ( Figure B-9 ). Negative correlations between TN and chl

- a likely reflect the complexities of the components of TN that are available for biological uptake. In certain locations, it seems likely that the majority of measured TN is comp osed of biological unavailable forms. Further analysis of relationships between nitrate+nitrite and chl-a support this interpretation (see estuary

- specific description of analyses for Tampa Bay).

The observed relationships between nutrients and chl- a confo rm with expectations, and provide a useful model for deriving nutrient criteria.

B

- 10Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries

Figure B -8. Estimates of the slope of the linear relationship between annual geometric mean TP and chl -a among different segments. Two panels from top to bottom show Northwest and Eastern estuaries. Open

circles: mean estimate of slope; vertical lines: estimated 90 % confidence intervals.

B

- 11Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries

Figure B -9

. Estimates of the slope of the linear relationship between annual geometric mean TN and chl -a among different segments. T wo panels from top to bottom show Northwest and Eastern estuaries. Open

circles: mean estimate of slope; vertical lines: estimated 90% confidence intervals.

B

- 12Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries



### B.5 **Computing Upper and L**  **ower B**  **ounds of A**  **vailable D**  **ata**
As described in the previous sections, when criteria were derived using stressor-response relationships outside the range of observed geometric mean concentrations, final criteria are based on either the upper or lower bound of the available data. Upper and lower bounds were quantified as the 10 th and 90 th percentiles of the available annual geometric means for a given analysis. In a few segments, too few distinct geometric mean concentrations for a particular parameter (i.e., < 10) were available to reliably estimate a 10 th or a 90 th percentile. I n these segments, upper and/or lower bounds were not computed.



### B.6 **Nutrient T**  **argets for Downstream Protective Value**  **C alculation** 

A mixing/dilution model wa s used to calculate the freshwater downstream protective values ( DPVs ) necessary to achieve nutrient crite rion values specified within different estuary segments. The mixing/dilution model is based on the premise that changes in nutrient concentrations starting from freshwater tributaries and through an estuary can be approximated as a conservative mixing process. Using this model, one can estimate the contributions of seawater and freshwater to any water quality sample, based on the observed salinity in the sample, freshwater salinity, and the salinity of seawater. With estimates of nutrient

concentrations in seawater, one can also predict freshwater nutrient concentrations that are necessary to achieve targeted nutrient concentrations at estuarine salinities.

One key component of this analysis is an estimate of TN and TP concentrations in seawater. To derive seawater concentrations of TN and TP, EPA examined available salinity data and station locations to identify stations that were most likely to reflect seawate r conditions ( Figure B-10 ). Based on this initial examination, EPA determined that data collected at the estuary pass of the St. Lucie estuary best characterized seawater salinity and nutrient concentrations.

In the selected estuary pass station of St. Lucie, nutri ent concentrations and salinity varied substantially over time, as expected, given different freshwater flow rates. A negative relationship between salinity and nutrient concentrations was observed, as one would expect, given that increased salinities indi cate increased dilution by seawater ( Figure B-11). At the highest salinities (> 35 PSU) TP concentrations converged to a single value (TP = 0.03 mg/L), which provided an estimate of seawater concentrations of TP. However, TN still exhibited substantial variability at these high salinities. This variability can likely be attributed to contributions of high salinity/high TN from side channels (see Figure B-10 ). Because of the potential effects of non-seawater contributions to TN, the minimum TN concentration observed between 34 and 36 PSU was used as an estimate of seawater TN (TN = 0.14 mg/L).

B

- 13Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries

Figure B- 10

. Map of station locations in St. Lucie estuary. Stations labeled in order of maximum observed sa linity (highest salinity is numbered 1). Station circled in red is located in the estuary pass.

Figure B- 11

. Observed salinity and nutrient concentrations at estuary pass of St. Lucie.

B

- 14Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries

For this analysis, the salinity of freshwater is assumed to be 2.7 PSU and that of seawater is

assumed to be 36 PSU

.

Given an estimate of seawater salinity and nutrient concentrations and a targeted nutrient criterion value and long- term salinity value for a given segment, one can compute a freshwater DPV by simply fitting a straight line through the seawater value and the criterion value, and calculating a nutrient concentration at 2.7 PSU . This calculation can be written as follows for

TN:

TN
cr it − TN
sea DPV = ( 2.7

− S
sea ) + TN
sea S

seg − S
sea

w here TN crit is the proposed TN criterion for a particular segment, TN
sea is the seawater concentration of TN, S seg is the long- term salinity in the segment of interest, and S
sea i s the salinity of seawater (taken to be 36 PSU ). The approach is illustrated graphically in Figure B-12.

Figure B- 12

. Illustration of calculation of DPVs using a mixing/dilution relationship for Indian River Lagoon

. Black
diamond shows the estimated conditions in seawater. Green circles show proposed TP criteria for different
segments. Green triangles show DPVs computed for each segment. Open circles show long

- term average salinity
and TP and different stations in the estuary.



### B.7 **R eferences** 

FDEP . 2010. IWR Stations Run 40

. http://www.dep.state.fl.us/water /basin411/download.htm

. Last updated April 15, 2010.

Gelman, A., and J. Hill. 2009. Data Analysis Using Regression and Multilevel/Hierarchical

Models . Cambridge University Press, New York.

Holmes, R.W. 1970. The Secchi disk in turbid coastal zones. Limnology and Oceanography.

15:688-694.

B

- 15Technical Support Document for U.S. EPA’s Proposed Rule for Numeric Nutrient Criteria,

Volume 1 Estuaries

R Development Core Team. 2009. R: A language and environme nt for statistical computing. R Foundation for Statistical Computing, Vienn a, Austria. ISBN 3 -900051-07-0. http://www .R-project.org

.

Wood, S.N., and N.H. Augustin. 2002. GAMs with integrated model selection using penalized regression splines and applications to environmental modelling. Ecological Modelling

157:157-177.

B

- 16