TSD for U.S. EPA’s Proposed Numeric Nutrient Criteria for FL Inland Surface Fresh Waters

Appendix B-6: Statistical Analysis of Ambient Water Quality Data 

in Florida Streams by FDEP’s Nutrient Regions 
TSD for U.S. EPA’s Proposed Numeric Nutrient Criteria for FL Inland Surface Fresh Waters

B6-1 

Appendix B-6. Statistical Analysis of Ambient Water Quality Data in Florida 

Streams by FDEP’s Nutrient Regions 

The United States Environmental Protection Agency’s (U.S. EPA) Nutrient Criteria Technical 

Guidance Manual: Rivers and Streams (USEPA 2000a) describes several general approaches, 

with certain options, that can be used to develop numeric nutrient criteria for streams. In all 

cases, consistent with the Clean Water Act (CWA), the objective is to determine a level of 

nutrients that will provide protection of aquatic life both in-stream and downstream of a point of 

application. The approach most often preferred is to quantify a stressor-response relationship 

between excess nutrients and adverse effects on aquatic life and then use that relationship to 

establish effects-based criteria. The necessary observed dose-response relationship could be 

described by a model (e.g., trophic state classification, regional predictive model, nutrient 

sensitive biocriteria), which, in turn, would link nutrient concentrations to the relative risk of 

environmental harm. Nutrient criteria based on a cause-effect relationship between nutrients and 

valued ecological attributes could thus be considered directly linked to maintaining designated 

uses. Such criteria might be scientifically defensible but have proved to be elusive. While the 

underlying relationship certainly exists, in nature it has often been difficult to quantify because 

of the complexity of nutrient dynamics and the influence of interfering factors. 

Without data and information to quantify biological dose-response relationships for nutrients, 

U.S. EPA recommends the reference condition approach as an alternative. With that approach, a 

reference condition in terms of nutrient concentration is determined empirically from a statistical 

distribution of nutrient data that can be assumed to inherently protect aquatic life. Under U.S. 

EPA’s reference condition approach, one option involves setting criteria on the basis of an 

inclusive distribution of nutrient values obtained from minimally disturbed reference sites in a 

designated ecoregion (based on climate and geology). In such a case, U.S. EPA recommends 

using an upper percentile value of this distribution to represent a level of nutrient concentration 

that will inherently protect aquatic life. This methodology will ensure that nutrient 

concentrations associated with biologically healthy, well-balanced communities would be 

considered acceptable as protective of those communities. However, the approach is an inference 

model and does not definitively demonstrate that exceeding the threshold established by the 

distribution of reference sites results in harm (impairment) to the aquatic life in a particular 

waterbody. Multiple factors can strongly influence the expression of biological responses to 

nutrients across waterbodies, such as water velocity, residence time, availability of the other 

nutrients, presence of grazers, availability of light (because of tree cover and/or water 

transparency), and availability of suitable habitat. The minimally affected reference water

approach identifies concentrations that are presumed to be inherently protective of the waterbody 

because these concentrations are associated with demonstrated minimally disturbed systems with 

healthy biological communities 

Also under U.S. EPA’s reference condition approach, another option can be used to extrapolate a 

reference condition of nutrient concentration that can be assumed to inherently protect aquatic 

life. In those situations where a cause-effect relationship cannot be numerically quantified (but is 

nonetheless firmly qualitatively established, e.g., nutrient impacts are documented) and where 

known, documented reference sites are either unavailable or of insufficient number, U.S. EPA 

suggests an option that is often referred to as the all streams approach. In this application of U.S. 
TSD for U.S. EPA’s Proposed Numeric Nutrient Criteria for FL Inland Surface Fresh Waters

B6-2 

EPA’s reference condition approach, a distribution of nutrient data is assembled from all sites in 

an ecoregion (or, alternatively, a random selection of sites). Assuming the pool of sites in the 

distribution is large enough to represent all waters in the region and can be presumed to 

reasonably reflect the full range of ambient conditions with a disturbance gradient from least to 

most affected, the selection of a lower percentile of this distribution (e.g., 25th percentile) should 

provide a nutrient level that can be assumed to inherently protect aquatic life. Because of the 

intended scale of application and the availability of data at the time of publication, U.S. EPA 

used this method to develop and publish under CWA section 304(a) recommended nutrient 

criteria for streams and lakes in nutrient ecoregions IX, XII, and XIII. Although an all sites data 

distribution approach does not directly link nutrient concentrations to valued ecological attributes 

(healthy, well-balanced populations of flora or fauna), it does make the presumption that the 25th

percentile will closely approximate a reference condition for that geographic location. Several 

examples with data from actual reference sites have supported this assumption (USEPA 2000a; 

USEPA 2000b) 

The purpose of the analysis presented here is to evaluate data from all sites by the Florida 

Department of Environmental Protection (FDEP) nutrient region, not simply by reference site. 

The 25th percentile was published by U.S. EPA in guidance on developing nutrient criteria 

(USEPA 2000a; USEPA 2000b; USEPA 2001; USEPA 2007), and the 25th percentile of the 

entire site population was the basis of U.S. EPA’s recommended ecoregional criteria (USEPA 

2000c; USEPA 2000d). Although the 25th percentile was recommended, other percentiles are 

presented in this appendix because they might have application elsewhere. Benchmark site 

distributional statistics, computed elsewhere, are compared to the all streams statistics presented 

here. 

Data Processing Steps 

This section provides an overview of the data used and processing steps to compute the 25th

percentile statistics by FDEP nutrient region. 

 FDEP provided the all streams data set to EPA. The data were quality assurance (QA) 

reviewed and screened and served as the population of sites from which FDEP drew the 

benchmark data set. FDEP verified that these data, primarily from STORET, are of known or 

defensible quality and the data are data that are very representative of Florida’s flowing 

waters. The all streams data set contained 92,360 records corresponding to 7,373 sites. The 

latitude and longitude from the Impaired Waters Rule (IWR) database, Run 35 was used for 

44 all streams sites that did not have latitude and longitude in the FDEP-provided all streams

data set. 

 The updated site locations were entered into a geographic information system (GIS) along 

with the revised FDEP nutrient regions and Florida waterbody ID (WBID) coverage, also 

provided by FDEP. Sites were further attributed with the FDEP nutrient region and WBID. 

Removing sites that did not exist in one of the FDEP nutrient regions and sites that were not 

in a WBID left 6,383 sites, or 78,370 records, for further consideration. 

 One of the FDEP outlier analysis steps for benchmark sites was to remove observations 

greater than the upper adjacent value. When applied to the total phosphorus (TP) and total 

nitrogen (TN) data by FDEP nutrient region, 5% to 12% of the data would be removed 

depending on FDEP nutrient region and parameter. This approach was abandoned in lieu of a 
TSD for U.S. EPA’s Proposed Numeric Nutrient Criteria for FL Inland Surface Fresh Waters

B6-3 

simple screening threshold to remove erroneous data. The screening thresholds of 15 mg/L 

and 100 mg/L of TP and TN, respectively, were selected, and 18 TP and 20 TN observations 

were removed. The same threshold of 100 mg/L was applied to the other nitrogen parameters 

analyzed. Further, if any nitrogen parameter exceeded 100 mg/L, all nitrogen parameters for 

that sample were removed. Data from canals and samples with a specific conductivity 

exceeding 5,000 μS/cm were removed from the analysis so as to focus on freshwater streams 

and rivers. That process reduced the number of records from 78,370 to 65,311 for processing. 

 Total inorganic nitrogen (TIN) was computed as the sum of ammonia (NH3) plus nitrite (NO2) 

plus nitrate (NO3), and organic nitrogen (OrgN) was computed as Total Kjeldahl Nitrogen 

(TKN) minus NH3 only when both component parameters were reported. 

 The analyzed chl. a data are for suspended chlorophyll and are for raw and pheophytin 

corrected values. 

 Data were organized by WBID-Year, and observation counts were made in a manner identical 

to that used by FDEP’s analysis of benchmark sites. That is, whenever the TP observation 

count for a particular WBID-Year was less than four, all TP observations for that WBID-Year 

were dropped. A similar process for TN and all other nitrogen parameters was implemented. 

That process reduced the number of records from 65,311 to 51,366 for processing. 

 The TP, nitrogen, and chlorophyll parameters were transformed by taking the natural 

logarithm (e.g., LN(TP), LN(TN)). Log transformation converts data that are positively 

skewed to more symmetrically (closer to normally) distributed data. The mean of the 

transformed data was computed for each WBID-Year remaining in the analysis after the 

above screening steps. 

 The above analyses were performed using MS Access®. To check the computations, the 2005 

data from WBID 1443D were extracted from the original FDEP file and analyzed using a 

spreadsheet. The end results were compared to the results from MS Access® and found to be 

the same. Also, individual data for site 21FLGW_3549 were compared to the benchmark data 

set. All nitrogen and phosphorus results were identical for the 99 samples that were in 

common. The benchmark data set did have one additional sample (the most recent) that might 

be attributed to source data status, but it was not believed to be a significant issue for the 

current work. 

Nutrient Results 

Using the data processed as described earlier, the mean, standard deviation, and various 

percentiles were estimated by FDEP nutrient region (Tables B6-1 and B6-2). The means and 

standard deviations from Tables B6-1 and B6-2 were used to compute theoretical distributions 

assuming a lognormal distribution and were back-transformed (Tables B6-3 and B6-4). The 

percentiles shown in Table B6-1 were also back-transformed and are shown as “empirical 

computed distribution statistics” in Table B6-3 as well. 

TP distributional statistics are presented for the Bone Valley and North Central Regions (Figure 

B6-1) and for the Northeast, Panhandle, and Peninsula Regions (Figure B6-2). TN distributional 

statistics are presented for the Panhandle Region and the combined Northeast-North CentralPeninsula-Bone Valley Region (Figure B6-3). In Figures B6-1, B6-2, and B6-3, open symbols 

indicate percentiles computed from an assumed lognormal distribution while solid symbols 
TSD for U.S. EPA’s Proposed Numeric Nutrient Criteria for FL Inland Surface Fresh Waters

B6-4 

indicate those from the empirical distribution. When comparing the empirical and theoretically 

derived 25th percentiles (i.e., 0.236 mg/L and 0.227 mg/L TP for the Bone Valley from Table B6-

3), the maximum difference was 0.009 mg/L for TP and 0.029 mg/L for TN. The maximum 

difference for the other nitrogen parameters was 0.079 mg/L. 

Although little difference existed between the theoretical and empirical computed results, the 

theoretically computed percentiles from this analysis were used and compared to similarly 

computed values from the benchmark sites (Table B6-5). In all instances for all nutrient species, 

the 25th percentile from the all streams data was less than the 75th percentile (and therefore the 

90th percentile) from the benchmark data set. In particular, the all streams 25th percentile TP was 

approximately 25% of the benchmark 90th percentile and 40% of the benchmark 75th percentile 

in the North Central and Panhandle Regions. The all streams 25th percentile TP ranged from 42% 

to 60% of the benchmark 90th percentile and 57% to 80% of the 75th benchmark percentile in the 

Bone Valley, Northeast, and Peninsula Regions. TN values for the all streams 25th percentile TN 

ranged from 51% to 53% of the benchmark 90th percentile and 67% to 69% of the benchmark 

75th percentile. The all streams 25th percentile TIN and OrgN ranged from 28% to 38% of the 

benchmark 90th percentile and 67% to 69% of the benchmark 75th percentile. The all streams 25th

percentile nitrate/nitrite (NOx) ranged from 13% to 18% of the benchmark 90th percentile and 

26% to 37% of the benchmark 75th percentile. 
TSD for U.S. EPA’s Proposed Numeric Nutrient Criteria for FL Inland Surface Fresh Waters

B6-5 

Table B6-1. Distributional Statistics Computed from the Natural Logarithm Transformed All Streams Data 

Observations Distributional Statistics Estimated from LN Trans

Parameter 

FDEP Nutrient Region 

Number 

of WBIDYears Number of WBIDs Mean LN Std Dev LN 10th %tile LN 25th %tile LN 50th %tile LN 

TP (mg/L) Bone Valley 555 142 -0.8694 0.8489 -1.84547 -1.48194 -0.95708 TP (mg/L) North Central 420 50 -1.6507 1.0598 -2.85363 -2.3291 -1.75383 TP (mg/L) Northeast 347 111 -2.3968 0.8153 -3.27757 -2.88301 -2.45331 TP (mg/L) Panhandle 435 167 -3.417 0.9884 -4.61254 -4.16749 -3.39228 TP (mg/L) Peninsula 1843 427 -2.1644 0.7407 -3.0213 -2.63774 -2.22592 TN (mg/L) Panhandle 404 146 -0.427 0.5999 -1.18593 -0.79297 -0.42644 TN (mg/L) NE-NC-Pen-BV 2902 702 0.16763 0.44259 -0.34988 -0.09828 0.14958 NOx (mg/L) Panhandle 429 164 -2.1851 1.5555 -4.57421 -3.10851 -2.00869 NOx (mg/L) NE-NC-Pen-BV 3054 718 -2.6932 1.3904 -4.4457 -3.76488 -2.79399 TKN (mg/L) Panhandle 417 153 -0.9832 0.6733 -1.70945 -1.35516 -1.05565 TKN (mg/L) NE-NC-Pen-BV 3053 716 -0.078 0.6005 -0.73685 -0.36267 -0.02878 NH3 (mg/L) Panhandle 432 168 -3.8099 0.9212 -4.83967 -4.4046 -3.9082 NH3 (mg/L) NE-NC-Pen-BV 3070 720 -3.2901 0.8493 -4.27856 -3.84872 -3.35899 OrgN (mg/L) Panhandle 407 153 -1.0882 0.6501 -1.82163 -1.44124 -1.16697 OrgN (mg/L) NE-NC-Pen-BV 2959 707 -0.1606 0.6033 -0.81399 -0.44129 -0.10173 TIN (mg/L) Panhandle 418 162 -1.6997 1.1814 -3.2562 -2.53151 -1.74731 TIN (mg/L) NE-NC-Pen-BV 2938 707 -1.9287 1.019 -3.1485 -2.68879 -2.02119 
TSD for U.S. EPA’s Proposed Numeric Nutrient Criteria for FL Inland Surface Fresh Waters

B6-6 

Table B6-2. Distributional Statistics Computed from the Natural Logarithm Transformed All Streams Data 

Observations Distributional Statistics Estimated from LN Tran

Parameter 

FDEP Nutrient 

Region 

Number 

of WBIDYears 

Number 

of 

WBIDs Mean LN

Std Dev 

LN 

10th %tile 

LN 

25th %tile 

LN 

50th %tile 

LN 

Bone Valley 128 375 0.6921 1.5871 -0.8557 -0.6931 0.3365 North Central 42 77 -0.4064 1.0975 -0.8557 -0.8557 -0.8557 Northeast 106 302 0.7960 1.5322 -0.8557 -0.6931 0.6864 Panhandle 150 360 0.1228 1.0592 -0.8557 -0.8557 -0.1625 

Corrected Chl. a (μg/L) 

Peninsula 366 1221 1.2804 1.6974 -0.8557 -0.3567 1.2940 

Corrected Chl. a Panhandle 150 360 0.1228 1.0592 -0.8557 -0.8557 -0.1625 (μg/L) NE-NC-Pen-BV 642 1975 1.0494 1.6786 -0.8557 -0.6931 0.8755 

Bone Valley 74 321 1.5383 1.5054 -0.3567 0.4055 1.3610 North Central 24 72 -0.2869 1.2013 -1.7720 -1.0498 -0.4620 Northeast 31 183 1.2276 1.5247 -0.6931 0.0062 1.4294 Panhandle 54 104 0.9944 0.5848 0.9163 0.9163 0.9163 

Chl. a (μg/L) 

Peninsula 229 1029 1.5916 1.9983 -0.6931 0.4054 1.9315 

Panhandle 54 104 0.9944 0.5848 0.9163 0.9163 0.9163 

Chl. a (μg/L) NE-NC-Pen-BV 358 1605 1.4472 1.8796 -0.6931 0.2401 1.6177 
TSD for U.S. EPA’s Proposed Numeric Nutrient Criteria for FL Inland Surface Fresh Waters

B6-7 

Table B6-3. Distributional Statistics Computed from the Back Transformed Results of the All Streams Data 

Parameter Region 10th %tile 25th %tile 50th %tile 75th %tile 

Distribution Statistics Computed from the Theoretical Lognormal Distribution Parameters TP (mg/L) Bone Valley 0.141 0.236 0.419 0.743 North Central 0.049 0.094 0.192 0.392 Northeast 0.032 0.053 0.091 0.158 Panhandle 0.009 0.017 0.033 0.064 Peninsula 0.044 0.070 0.115 0.189 Empirically Computed Distribution Statistics Bone Valley 0.158 0.227 0.384 0.700 North Central 0.058 0.097 0.173 0.316 Northeast 0.038 0.056 0.086 0.135 Panhandle 0.010 0.015 0.034 0.053 Peninsula 0.049 0.072 0.108 0.168 Distribution Statistics Computed from the Theoretical Lognormal Distribution Parameters Panhandle 0.302 0.435 0.652 0.978 NE-NC-Pen-BV 0.671 0.877 1.182 1.594 Empirically Computed Distribution Statistics Panhandle 0.305 0.452 0.653 0.897 

TN (mg/L) 

NE-NC-Pen-BV 0.705 0.906 1.161 1.525 

Distribution Statistics Computed from the Theoretical Lognormal Distribution Parameters Panhandle 0.015 0.039 0.112 0.321 NE-NC-Pen-BV 0.011 0.026 0.068 0.173 Empirically Computed Distribution Statistics Panhandle 0.010 0.045 0.134 0.359 

NOx (mg/L) 

NE-NC-Pen-BV 0.012 0.023 0.061 0.186 

Distribution Statistics Computed from the Theoretical Lognormal Distribution Parameters Panhandle 0.158 0.238 0.374 0.589 NE-NC-Pen-BV 0.428 0.617 0.925 1.387 Empirically Computed Distribution Statistics Panhandle 0.181 0.258 0.348 0.553 

TKN (mg/L) 

NE-NC-Pen-BV 0.479 0.696 0.972 1.313 
TSD for U.S. EPA’s Proposed Numeric Nutrient Criteria for FL Inland Surface Fresh Waters

B6-8 

Parameter Region 10th %tile 25th %tile 50th %tile 75th %tile 

Distribution Statistics Computed from the Theoretical Lognormal Distribution Parameters Panhandle 0.007 0.012 0.022 0.041 NE-NC-Pen-BV 0.013 0.021 0.037 0.066 Empirically Computed Distribution Statistics Panhandle 0.008 0.012 0.020 0.034 

NH3 (mg/L) 

NE-NC-Pen-BV 0.014 0.021 0.035 0.059 

Distribution Statistics Computed from the Theoretical Lognormal Distribution Parameters Panhandle 0.146 0.217 0.337 0.522 NE-NC-Pen-BV 0.393 0.567 0.852 1.279 Empirically Computed Distribution Statistics Panhandle 0.162 0.237 0.311 0.513 

OrgN (mg/L) 

NE-NC-Pen-BV 0.443 0.643 0.903 1.223 

Distribution Statistics Computed from the Theoretical Lognormal Distribution Parameters Panhandle 0.040 0.082 0.183 0.405 NE-NC-Pen-BV 0.039 0.073 0.145 0.289 Empirically Computed Distribution Statistics Panhandle 0.039 0.080 0.174 0.416 

TIN (mg/L) 

NE-NC-Pen-BV 0.043 0.068 0.132 0.288 
TSD for U.S. EPA’s Proposed Numeric Nutrient Criteria for FL Inland Surface Fresh Waters

B6-9 

Table B6-4. Distributional Statistics Computed from the Back Transformed Results of the All Streams Data 

Distribution Statistics Computed from the Theoretical Lognormal 

Parameters 

Parameter Region 10th %tile 25th %tile 50th %tile 75th %tile 

Bone Valley 0.261 0.685 1.998 5.827 North Central 0.163 0.318 0.666 1.396 Northeast 0.311 0.789 2.217 6.230 Panhandle 0.291 0.553 1.131 2.310 

Corrected Chl. a (μg/L) 

Peninsula 0.409 1.145 3.598 11.305 

Panhandle 0.291 0.553 1.131 2.310 

Corrected Chl. a (μg/L) NE-NC-Pen-BV 0.332 0.921 2.856 8.860 

Bone Valley 0.676 1.687 4.657 12.854 North Central 0.161 0.334 0.751 1.688 Northeast 0.484 1.220 3.413 9.545 Panhandle 1.278 1.822 2.703 4.010 

Chl. a (μg/L) 

Peninsula 0.379 1.276 4.912 18.905 

Panhandle 1.278 1.822 2.703 4.010 

Chl. a (μg/L) NE-NC-Pen-BV 0.382 1.197 4.251 15.104 
TSD for U.S. EPA’s Proposed Numeric Nutrient Criteria for FL Inland Surface Fresh Waters

B6-10 

Table B6-5. Comparison of Distributional Statistics from the All Streams Data and Alternative Benchmark Distr

Theoretical Percentiles Computed 

from Mean LN and Std Dev LN from 

All Streams Data 

FDEP 

calculation 

from 

Benchmark_

Regions_051

109.xlsx 

Scenario 1: 

All 4 outlier 

WBIDs 

removed 

(1653_, 

1807c_ 

1811_, and 

3401_) 

Scenario 2: 2 

outlier WBIDs 

removed 

(1653_ and 

3401_); 1807c_ 

and 1811_ 

retained in 

Bone Valley 

Scen

Benc

WBI

distri

WBI

rem

Parameter Region 

10th

%tile 

25th

%tile

50th

%tile

75th

%tile

90th

%tile

75th

%tile

90th

%tile

75th

%tile 

90th

%tile

75th

%tile 

90th

%tile 

75th

%tile 

TP (mg/L) Bone Valley 0.141 0.236 0.419 0.743 1.244 0.75 1.168 0.34 0.451 0.41 0.559 0.41 

North Central 0.049 0.094 0.192 0.392 0.746 0.235 0.359 0.216 0.322 0.216 0.322 0.235 Northeast 0.032 0.053 0.091 0.158 0.259 0.066 0.101 0.066 0.101 0.066 0.101 0.066 Panhandle 0.009 0.017 0.033 0.064 0.116 0.043 0.069 0.043 0.069 0.043 0.069 0.043 Peninsula 0.044 0.070 0.115 0.189 0.297 0.088 0.116 0.088 0.116 0.088 0.116 0.088 

TN (mg/L) Panhandle 0.302 0.435 0.652 0.978 1.407 0.63 0.82 0.63 0.82 0.63 0.82 0.63 

NE-NC-Pen-BV 0.671 0.877 1.182 1.594 2.085 1.31 1.73 1.31 1.73 1.31 1.73 1.31 

TIN (mg/L) Panhandle 0.040 0.082 0.183 0.405 0.831 0.18 0.29 0.18 0.29 0.18 0.29 0.18 

NE-NC-Pen-BV 0.039 0.073 0.145 0.289 0.536 0.13 0.20 0.13 0.19 0.13 0.20 0.13 

OrgN (mg/L) Panhandle 0.146 0.217 0.337 0.522 0.775 0.45 

NE-NC-Pen-BV 0.393 0.567 0.852 1.279 1.845 1.12 

NOx (mg/L) Panhandle 0.015 0.039 0.112 0.321 0.826 0.15 

NE-NC-Pen-BV 0.011 0.026 0.068 0.173 0.402 0.07 Bone Valley 0.261 0.685 2.00 5.83 15.3 North Central 0.163 0.318 0.666 1.40 2.72 Northeast 0.311 0.789 2.22 6.23 15.8 Panhandle 0.291 0.553 1.13 2.31 4.34 

Corrected Chl. a (μg/L) 

Peninsula 0.409 1.15 3.60 11.3 31.7 

Corrected Panhandle 0.291 0.553 1.13 2.31 4.40 Chl. a (μg/L) NE-NC-Pen-BV 0.332 0.921 2.86 8.86 24.5 
TSD for U.S. EPA’s Proposed Numeric Nutrient Criteria for FL Inland Surface Fresh Waters

B6-11 

Figure B6-1. TP distributional statistics for the Bone Valley and North Central Regions. 

Figure B6-2. TP distributional statistics for the Northeast, Panhandle, and Peninsula Regions. 
TSD for U.S. EPA’s Proposed Numeric Nutrient Criteria for FL Inland Surface Fresh Waters

B6-12 

Figure B6-3. TN distributional statistics for the Panhandle Region and the combined NortheastNorth Central-Peninsula-Bone Valley Region.