Document ID: EPA-HQ-OW-2009-0596-0165
Agency: epa
Document Type: Supporting & Related Material
Title: 
Posted Date: 2010-01-15T05:00Z

Florida Department of Environmental Protection Review of SPARROW:  
How useful is it for the purposes of supporting water quality standards development?

The Florida Department of Environmental Protection (FDEP) conducted a review of the SPARROW model with the intent of understanding its technical strengths and limitations for water quality management purposes.  FDEP utilized State modeling and water quality assessment expertise, largely developed through implementation of the Total Maximum Daily Load (TMDL) programs, which provides a sound basis for the level of technical veracity necessary to support regulatory actions such as setting TMDLs or water quality standards.
General Comments
FDEP believes SPARROW is a useful tool for assessment of general patterns of major nutrient sources in watersheds.  However, FDEP observed some limitations in the use of SPARROW for developing scientifically defensible numeric nutrient values that would be used to drive regulatory decisions. We understand that this is not new information as the USGS acknowledged that the regional model was not intended to be used for site-specific application in regulatory programs without significant development of site-specific information, and also acknowledged that the model had geographic limitations, such as in South Florida and areas with high groundwater recharge.  Beyond that, FDEP recognizes that the SPARROW model used only four general source inputs and was calibrated across a large geographic area.  This coarse spatial resolution resulted in large error bars associated with the model output for any specific geographic area.  The FDEP also understands that SPARROW was calibrated to only one year's worth of data, and was not validated by testing data from other years.  These above factors simply raise too many questions regarding output results and offer too much legal vulnerability regarding the technical veracity of the decisions supported by the model. 
Because of these technical vulnerabilities, calculating the presumed allowable loading using the model cannot be scientifically supported and leads to errors when compared to loads necessary to protect the designated use of various estuaries.  EPA offered an approach using SPARROW to estimate natural background loads as a basis for deriving a protective target. There are several examples where the proposed values for the allowable loading are considerably lower than what is actually necessary to protect the estuaries (e.g, Apalachicola Bay, Apalachee Bay, St. Marys River).  This error is probably largely due to the simplified source input variables to the model, coupled with the limitations on how the model was calibrated but not validated and the challenges of estimated background loads, but also raises questions about EPA's proposal to set the protective target at the midpoint between current and background loading..
FDEP also notes that, when comparing the protective long-term averages calculated using SPARROW (and the downstream calculator) to the proposed benchmark criteria for streams, it is important to take the expression of the criteria into account.  To be in compliance with the criteria (expressed as an annual geometric mean not to be exceeded more than once in a 3-year period), the long term mean nutrient concentration must be lower than the 90[th] percentile value of the reference site population (see Table 1).  For example, the long-term average TN concentration in Panhandle tributaries would need to be 0.56 mg/L in order to meet the proposed standard of an annual geometric mean of 0.82 mg/L not to be exceeded more than once in a 3-year period, and this value is well below the protective long-term averages calculated by Hagy and Green.  Therefore, we conclude that the Department's proposed Panhandle bioregion nitrogen criterion would be considerably lower than what is necessary to protect Apalachicola Bay. 
In summary, FDEP believes that the SPARROW model is not appropriate at this time for use as the basis for regulatory decisions.  A more appropriate approach, short of further calibration and validation of models on site specific watersheds, would be to use local scientific judgment and analysis for each estuary to estimate protective loads, like was done for the Tampa Bay estuary.
Table 1.    Long-term average total nitrogen concentrations necessary to achieve the proposed stream benchmark criteria at least 90% of the time.   
Parameter
                                Nutrient Region
                         Magnitude of Draft Criterion
Required Long-term Concentration to Achieve the  1 in 3 Year Test with 90% Confidence
                           Long-term Geometric Mean
TN (mg/L)
Panhandle
                                     0.82
                                     0.56

NE-NC-Pen-BV
                                     1.73
                                     1.11

Specific Comments:
1. The "atmosphere N load to the estuary" estimate from SPARROW needs to be verified against site-specific examples to assess the accuracy of these predictions.  Also the atmospheric (background) contribution needs to take into account the range of loads rather than using a static (detrended 2002) value.

2. Chapter 14 of EPA's NCD Technical Document (EPA 2009) notes that "Interim loading targets could not be developed for 5 estuaries in south Florida because required information related to TN loading could not be obtained for these systems in the time available."  The technical documentation (Hoos et al., 2008) for the specific application of SPARROW being used here has the following caveat:  "Nutrient conditions measured in streams affected by substantial influx or outflux of water and nutrient mass across surface-water basin divides do not reflect nutrient source and transport conditions in the topographic watershed; therefore, inclusion of such streams in the SPARROW modeling approach is considered inappropriate.  River basins identified with this concern include south Florida (where surface-water flow paths have been extensively altered) and the Oklawaha, Crystal, Lower Santa Fe, Lower Suwannee, St. Marks, and Chipola River basins in central and northern Florida (where flow exchange with the underlying regional aquifer may represent substantial nitrogen influx to and outflux from the surface-water basins)."  FDEP understands the source of these limitations and would suggest using a statistical approach using actual data, versus a modeling approach, to estimate protective loadings to estuaries.  
3. The Hoos et al., 2008 document states that "For local-scale interpretations use of loads estimated for actual time periods and employing a more detailed regression analysis, such as stepwise linear regression and consideration of additional explanatory variables, is suggested."  FDEP agrees that more site specific regression and source variables would increase the veracity of the estimates.   

4. Subsection 4.1.3 (EPA 2009) states that "across the 13 estuaries in Florida, we find that atmospheric N loading levels vary from 20% to 60% of the current TN loading (Table 1).  Thus, allowable TN loads that could ensure protection of designated use in downstream estuarine waters must be constrained between the loading to estuarine waters resulting from the atmospheric load and the current total load to estuarine waters from all sources."  This makes the presumption that current load may not be protective when there are number of estuaries currently in good health across the State.  Additionally, the source of atmospheric load estimates are calibrated to a limited number of stations that may be appropriate on a large geographic scale, but loses a large degree of technical merit when then applied to a relatively small geographic area.  Estimating natural background using only atmospheric loads appears to eliminate loads from natural flora and fauna (e.g., leaf litter from the extensive forests throughout Florida).  It may be a worthwhile to exercise to verify this estimate by comparing such them to actual nutrient loads in relatively undisturbed areas to assess its accuracy.

5. Subsection 14.1.5 (EPA 2009) states that "In Florida, recommended reductions from TMDLs and PLRGs range from 2% to 63% (mean 36%, median 38%).  Larger reductions are recommended where anthropogenic loading has increased loads by the largest margin.  Based on the information herein, we suggest that in the absence of more specific information it may be appropriate to reduce the anthropogenic contribution of TN load to estuaries by a constant fraction.  For example, by reducing the anthropogenic fraction of TN loads by 50% results, on average, in a reduction of overall loading by 30%, similar to the loading reductions determined via TMDLs and PLRGs in Florida."  We believe that more specific information is available for many of the 13 estuaries evaluated in Section 14 and that there needs to be a more comprehensive assessment comparing results of that information to that set forth in Chapter 14.  DEP staff have prepared or in the process of preparing such assessments for several Florida estuaries (i.e, Apalachicola Bay, St. Johns River, Tampa Bay).  Assessments for Apalachicola Bay and the St. Johns River are attached.

6. Table 3 (page 192), which contains SPARROW watershed areas and "Watershed" areas for the 13 estuaries, states that "The difference between the SPARROW-watershed area and the total watershed area reflects areas that are not hydrologically-resolved by the ERF1_2 file." It is not clear how the total watershed area is calculated and what is meant by this statement.  We note that these two watershed area estimates differ by from 1-70 percent and that the "Watershed" area is typically larger than the SPARROW watershed area.  We assume the load shown in the table is calculated from the SPARROW watershed area.  Furthermore, the text on page 193 indicates that "model estimates for some river basins in central and northern Florida ... may be affected by unmodeled nitrogen flux to and from the regional aquifer (Hoos et al. 2008).  We note that the Suwannee River, which is included in the list of such basins, is being included in this assessment.  This is a further complication that suggests that SPARROW may not be appropriate for supporting regulatory decisions at a smaller geographic scale at this time.
  
7. The SPARROW estimate of watershed TN load for Apalachicola Bay detrended to 2002  (~ 16 million kg/year) is within the range of estimates (~ 10 to ~31 million kg/year) for Apalachicola Bay given by Frick et al., 1996.  However, it is difficult to evaluate model performance without comparing the rainfall conditions in the basin in 2002 to the conditions in the 1974-1990 time period that Frick et al. reported on.  This comparison should be done for other estuaries for which there are reported loading values to assess the accuracy of the SPARROW model predictions.

8. The "interim allowable TN load protective of downstream use" as it is currently being derived (midpoint between current loading and estimated background loading) fails to make a connection to ecological health of estuaries and presumes that current load is not protective of estuaries across the State, which is false.  FDEP would recommend using site-specific loads derived through the TMDL, National Estuarine Program, or other scientifically-defensible estimate of the assimilative capacity of the estuaries for this analysis.

9. Page 187 of EPA (2009) states that "we applied both Lm models to all 13 of the estuaries in Florida.  In 10 estuaries, the Lm vs. w model provided a result that was between the atmospheric N loading level and current TN loading, whereas the Lm vs. qs model provided estimates for 11 estuaries that fell within that range, given the error associated with the Lm estimates.  For those estuaries where the model provided a value within the range between current loading and atmospheric loading, we considered it a potentially useful reference point for determining interim loading targets.  Where the model predicted a loading maximum outside that range, we conclude that an alternative approach is needed ." FDEP is still not clear what is intended by this.  However, it is important to note that each approach has its limitations and, as a regulatory agency responsible for decisions with respect to nutrient loading, FDEP always prefers to rely on site specific information that is calculated in a scientifically defensible manner over more generalized approaches, specifically with regards to estuaries.  

   A comparison of the maximum loading limits for the 13 estuaries calculated by the two versions of this model indicates significantly dissimilar results (Figure 1).  Of additional concern is that the maximum limit for both models is below the SPARROW-predicted background load for Apalachee Bay and Choctawhatchee Bay, and the  ~f(w) model estimate for the St. Mary's River is also below SPARROW-predicted background load.  Conversely, the maximum limit for the ~f(w) model is above the SPARROW-predicted current load for the Indian River, St. Andrew Bay and Perdido Bay, while the maximum limit for the ~f(qs) model is above the SPARROW-predicted current load for the Tampa Bay, Sarasota Bay and Pensacola Bay.   These disparities in the respective predictive abilities of the SPARROW and S&LMLL models need to be addressed before use for regulatory purposes.
                                          
Figure  1.  Comparison of Steward and Lowe Maximum Loading Limits.

10. While FDEP recommends that EPA base the downstream protection target for Tampa Bay on the federally approved Total Nitrogen TMDL for the bay, if EPA prefers developing a flow-weighted average concentration of TN for the downstream protection target, we recommend EPA use the TBEP dataset because the TMDL was also based on the load estimation methods of TBEP.  Table 2 presents a comparison between the GED SPARROW based load and the TBEP load estimates for Tampa Bay, and Table 3 provides the flow weight mean concentrations used to develop the TBEP estimates.  The SPARROW based loads are higher than the TBEP estimates.  Since the Tampa Bay TMDL is based on a hold the line strategy (because the Bay is meets the established chlorophyll a and seagrass targets), both estimates of current loads could be considered protective of downstream waters.  Based on site specific analysis in Tampa Bay, no additional load reductions are warranted for Tampa Bay since the loading is already below the SPARROW estimate.    

11. The data used by USGS for TN concentrations has not yet been received and reviewed; however there appear to be numerous flow stations that have little concentration data over the period of record used. The estimation methods used to fill these data gaps require further analysis.  The data period used by SPARROW SAGT (1974-2005) may not be reflective of current water quality conditions in the Tampa Bay watershed.  If the intent of the approach is to evaluate whether the proposed freshwater criteria are protective of downstream waters, there needs to be a clear demarcation of the water quality data used to characterize the fresh- and estuarine areas.  Tidal tributaries themselves are often a `pinch-point' for nutrient load-ecological response in a system, and would seem to be more responsive to load and duration rather than an annual flow-weighted average concentration.

12. DEP has prepared an assessment of the protectiveness to the Apalachicola Bay estuary of the state's proposed benchmark stream criteria for the Panhandle.  This assessment included an evaluation of the SPARROW model estimated loadings to the Bay.  The key conclusions of this analysis included:
         a)        Available nutrient data do not show an increasing trend, and recent chlorophyll a data indicate that Apalachicola Bay is not impaired for nutrients.

         b)       While the SPARROW model estimated average TN concentration for the base year (0.67 mg/L) compared favorably with the period of record average concentration of 0.74 mg/L TN, the averages do not take into account the natural variability in nitrogen loads and concentrations reaching Apalachicola Bay, which varied from under 5 million kg/year to almost 22.5 million kg/year.  Given this variability, the target load/concentration would be considered very conservative, particularly given that nutrient impairment is not evident in Apalachicola Bay throughout this range of nitrogen loads.
  
         c)       It should be noted that to achieve the Department's proposed reference site-based criteria (no more than one yearly exceedance in a three year period), the actual long term mean nutrient concentration must be substantially lower than the 90[th] percentile value of the reference site population. For example, using a probabilistic approach to calculate the long-term average nutrient concentrations necessary to achieve the proposed stream benchmark criteria at least 90% of the time in tributaries to Apalachicola Bay, the Department determined that the long-term average TN concentration in these tributaries would need to be 0.56 mg/L.  This value is well below the protective long-term averages calculated by Hagy and Green.  Therefore, we conclude that the Department's proposed Panhandle bioregion nitrogen criterion would be protective of Apalachicola Bay.
            
13. DEP also prepared an evaluation of the protectiveness of the proposed stream criteria to the Lower St. Johns River (attached) and the proposed benchmark-based stream TP and TN criteria and associated long term average concentrations are similar to the TMDL based nutrient concentrations in the Lower St. Johns River.  Based on this comparison, the proposed benchmark-based criteria for streams are inherently protective.

14. Department modeling staff have the following questions/comments/recommendations, based on their initial review of the SAGT model:
	
     a)       The Florida database of NPDES facilities includes over 100 facilities that are not included in the SAGT model.  Additional review is warranted to determine whether some of these facilities represent significant nutrient loads.  Similarly, approximately 65 facilities in the SAGT model had zero nutrient loads, and some of these facilities may have flow and concentration data available and loads could be calculated.

     b)       Some NPDES facilities have multiple discharges to different waterbodies and it was not clear how this was addressed in the model.  For example the Iron Bridge facility has discharges to both the Little Econhatchee and the St. Johns River, and it seemed to be input as a single discharge.

     c)       There are a number of facilities that have surface discharges as well as reuse, and it is not clear how the reuse portion of the discharge was addressed in the model, which could have significant implications on the accuracy of the point source loads.

     d)       Estimated atmospheric deposition may be significantly underestimated because dry deposition of nitrogen was not included.  Based on some previous USGS reports, dry deposition of nitrogen may equal or exceed wet deposition was not considered.  It was also not clear whether atmospheric deposition was directly applied to the surfaces of larger streams and/or lakes.

     e)       Nitrogen fixation by blue green algae can be a significant source of nitrogen in a number of waterbodies and was not considered as a source in the model.

     f)       Only three level 1 land classes were considered in the model input, and it is unclear why all eight classes were not considered, particularly if fertilizer and other loads were distributed among some of the other land types.  Wetlands, scrub/shrub, or grasslands could represent a substantial fraction of the catchment area in many watersheds. 

     g)       It does not appear that the model took into account the effects of control structures, which would have significant impacts on flow and water quality.  There are also a number of gaging sites that are influenced by discharger flows and it is unknown whether any special adjustments were made, especially if discharger flows and loads varied during the period.

     h)       Given the variability in rainfall and flows over the period of record, some further evaluation of the use of 2002 as a base year with respect to rainfall and other long-term attributes may be warranted.

     i)       Many catchments in Florida (approximately 168) do not seem to have loads or reach nodes.

     j)       Some additional verification of mean annual stream velocities should be considered since it determines reach time of travel and influences first order decay calculations.

     k)       We recommend that EPA run the model with upstream concentrations set at the proposed criteria to evaluate whether the resultant load is consistent with the estimated allowable loading.  We would expect that the fraction delivered would be different (lower) when upstream concentrations are lower.

Table 2.  Comparison of SPARROW flows and loads to those used for TBEP loading estimates
 
                                Gaged Site (1)
                                    SPARROW
 
                                     TBEP
                                    System
                                  Number (2)
                                   Location 
                                  Data Yr (3)
                               TN Tot Ld_kg (4)
                                   meanq (5)
                                       
                                    Data Yr
                             TN Total Load (kg/yr)
Annual Ave Flow (cfs)
Hillsborough River 
 
                                   02303330
@ Morris Br Rd
                                     2002
                                                                        300,304
                                                                            282
 
                                                                             --
                                                                             --
                                                                             --

21FLHILL.137
@ Columbus Dr Bridge
                                     2002
                                                                        243,116
                                                                            606
 
                                                                             --
                                                                             --
                                                                             --

                                   02304500
over Reservoir Dam
                                      --
                                                                             --
                                                                             --
 
                                                                           2002
                                                                        247,830
                                                                            240

                                       

                                       
                                                                               
                                                                               

                                                                               
                                                                               
                                                                               
Alafia River
                                   02301500
@ Lithia
                                     2002
                                                                        508,003
                                                                            382
 
                                                                           2002
                                                                        357,854
                                                                            266

                                       

                                       
                                                                               
                                                                               

                                                                               
                                                                               
                                                                               
Manatee River

                                   02299550
Near Myakka Head
                                     2002
                                                                        127,908
                                                                            157
 
                                                                             --
                                                                             --
                                                                             --

                                       
From Reservoir
                                      --
                                                                             --
                                                                             --

                                                                           2002
                                                                        144,452
                                                                            122

                                       

                                       
                                                                               
                                                                               

                                                                               
                                                                               
                                                                               
Little Manatee River
                                   02300500
Near Wimauma
                                     2002
                                                                        382,964
                                                                            230

                                                                           2002
                                                                        226,442
                                                                            142

                                       

                                       
                                                                               
                                                                               

                                                                               
                                                                               
                                                                               
Horse Creek
                                   02297310
Near Arcadia
                                     2002
                                                                        417,108
                                                                            260
 
                                                                           2002
                                                                        398,634
                                                                            298

1)	Sites are USGS unless noted.
2)	21FLHILL.137 = EPCHC site #137.  Discharge estimate methods unknown.
3)	The model predictions of the mean annual load for the calibrated model are standardized to a single year referred to as the "base year" to give an estimate of the mean nutrient load that would have occurred in streams during that year if long-term mean annual flow conditions had prevailed. The base year used for the Southeast US analysis was 2002.
4)	TotLd_kg = Instream load transported past the downstream end of the reach, in kilograms (for monitored reaches, predicted load is adjusted to match observed load)
5)	meanq = long-term mean annual flow for site (period of record ?) (assumed cfs)

Table 3.  Flow-weighted average TN concentrations based on data used to develop TBEP loads.  Total nitrogen were data collected by EPC at stations 105 (Hillsborough River) and 114 (Alafia River) began in 1981.
   
1981-2004
2003-2007
Hillsborough River at Dam
1.10 mg/L
0.91 mg/L
Alafia River at Lithia
2.38 mg/L
1.54 mg/L