EXTERNAL PEER REVIEWER COMMENTS

on

REVIEW OF DRAFT EPA REPORT, ‘ANALYSIS OF THE BROWN TROUT SELENIUM 

TOXICITY STUDY PRESENTED BY FORMATION ENVIRONMENTAL AND REVIEWED BY 

U.S. FISH AND WILDLIFE SERVICE’

Prepared for:

U.S. Environmental Protection Agency

Health and Ecological Criteria Division

Office of Science and Technology

Office of Water

Washington, DC

Prepared by

Great Lakes Environmental Center

Columbus, Ohio

June 2014
ii

TABLE OF CONTENTS

1. CHARGE TO PEER REVIEWERS ................................................................................................ 1

1.1. Regulatory Context .......................................................................................................................1

1.2. Background on Development of this Document...........................................................................1

1.3. Post-Peer Review Note (2014)......................................................................................................2

1.4. Charge Questions..........................................................................................................................2

1.5. Reviewers......................................................................................................................................4

2. CHARGE QUESTIONS, REVIEWERS COMMENTS AND EPA RESPONSE TO 

COMMENTS................................................................................................................................... 4

2.1. Question 1: Endpoints...................................................................................................................4

2.1.1. Peer Reviewer 1 ....................................................................................................................4

2.1.2. Peer Reviewer 2 ....................................................................................................................4

2.1.3. Peer Reviewer 3 ....................................................................................................................6

2.1.4. Peer Reviewer 4 ....................................................................................................................7

2.1.5. Peer Reviewer 5 ....................................................................................................................7

2.1.6. Peer Reviewer 6 ....................................................................................................................8

2.2. Question 2: Examination of uncertainty caused by loss of individuals in aquaria that 

overflowed, via best case and worst case assessment...............................................................................8

2.2.1. Peer Reviewer 1 ....................................................................................................................8

2.2.2. Peer Reviewer 2 ..................................................................................................................10

2.2.3. Peer Reviewer 3 ..................................................................................................................10

2.2.4. Peer Reviewer 4 ..................................................................................................................10

2.2.5. Peer Reviewer 5 ..................................................................................................................10

2.2.6. Peer Reviewer 6 ..................................................................................................................11

2.3. Question 3: Examination of uncertainty caused by compiling separate counts for each type of 

deformity.................................................................................................................................................11

2.3.1. Peer Reviewer 1 ..................................................................................................................11

2.3.2. Peer Reviewer 2 ..................................................................................................................11

2.3.3. Peer Reviewer 3 ..................................................................................................................12

2.3.4. Peer Reviewer 4 ..................................................................................................................12

2.3.5. Peer Reviewer 5 ..................................................................................................................16

2.3.6. Peer Reviewer 6 ..................................................................................................................16

2.4. Question 4: Possible use of FWS-calculated EC10 of 6.5 mg/kg in the derivation of a national 

criterion...................................................................................................................................................17

2.4.1. Peer Reviewer 1 ..................................................................................................................17

2.4.2. Peer Reviewer 2 ..................................................................................................................20

2.4.3. Peer Reviewer 3 ..................................................................................................................20

2.4.4. Peer Reviewer 4 ..................................................................................................................20

2.4.5. Peer Reviewer 5 ..................................................................................................................21

2.4.6. Peer Reviewer 6 ..................................................................................................................21

2.5. Question 5: Use of an EPA-calculated brown trout EC10 of 16.76 mg/kg in a Species 

Sensitivity Distribution for derivation of a national criterion.................................................................22

2.5.1. Peer Reviewer 1 ..................................................................................................................22
iii

2.5.2. Peer Reviewer 2 ..................................................................................................................28

2.5.3. Peer Reviewer 3 ..................................................................................................................29

2.5.4. Peer Reviewer 4 ..................................................................................................................30

2.5.5. Peer Reviewer 5 ..................................................................................................................31

2.5.6. Peer Reviewer 6 ..................................................................................................................32

3. GENERAL COMMENTS BY REVIEWERS OTHER THAN RESPONSE TO CHARGE

QUESTIONS ................................................................................................................................. 32

3.1. Peer Reviewer 1 ..........................................................................................................................32

3.2. Peer Reviewer 3 ..........................................................................................................................33

3.3. Peer Reviewer 5 ..........................................................................................................................33

3.4. Peer Reviewer 6 ..........................................................................................................................34

4. REFERENCES .............................................................................................................................. 38
1

1. CHARGE TO PEER REVIEWERS

The document reviewed a draft document titled “Analysis of the Brown Trout Selenium Toxicity Study 

Presented by Formation Environmental and Reviewed by U.S. Fish and Wildlife Service.”

1.1. Regulatory Context

EPA undertook this review because the information in the document under review may influence the 

derivation of a national water quality criterion for selenium. Such water quality criteria are published as 

recommendations under the Clean Water Act, to serve as guidance for state adoption into standards. EPA 

criteria recommendations are not regulations; rather, the state standards are regulations. States may adopt 

EPA criteria values, more stringent values, or where a scientifically reasonable rationale for their 

protectiveness is provided, less stringent values.

EPA criteria are not merely desirable goals. EPA must simultaneously defend its aquatic life criteria as 

having a level of stringency that is both necessary and adequate to protect aquatic life values – that is, as 

being neither overprotective nor underprotective.

1.2.Background on Development of this Document

EPA’s currently recommended national criterion, published in 1987, is a water concentration, 5 μg/L, 

derived from observations of a single site, Belews Lake, NC. Using toxicity data involving Se exposure 

via the diet, either from lab or field exposures, EPA published a draft criterion expressed as the Se 

concentration fish whole-body tissue in 2004, and requested public comment. Based on comments, EPA 

decided that the Se concentration in fish eggs or ovaries would be a more suitable basis for its future 

criterion. EPA believes that the recommendations of the SETAC 2009 selenium workshop support such 

an approach.

A number of studies, most of them fairly recent, are available for deriving an egg/ovary criterion. Like 

the brown trout study that is the subject of the work under review, the recent studies of maternal transfer 

of Se from adult females to their eggs involve exposure of adult female fish in the field, and observation 

of the health of the offspring in the lab.

EPA originally examined the maternal-transfer brown trout study, performed by ENSR and reported by 

Formation Environmental, as part of its development of a yet unpublished national recommended 

criterion for selenium, expressed as the concentration of selenium in the eggs or ovaries of fish. EPA’s 

original interpretation of the results reported by Formation yielded an EC10 that made brown trout (Salmo 

trutta) the most sensitive fish genus and species for reproductive effects, with EC10 lower than EPA’s 

calculated EC10s for Oncorhynchus, Salvelinus, Lepomis, Micropterus, and Esox. 

Without concern about any minor differences between EPA’s EC10s and those determined by the SETAC 

2009 selenium workshop (Janz et al. 2010), it is noted that for the eight species presented in the Janz et al. 

(2010) review, the thresholds for reproductive effects fall into a narrow band of 17 – 24 mg/kg (dry 

weight) in eggs. On the other hand, in a report published in January 2012, the U.S. Fish and Wildlife 

Service, raising certain issues not previously brought to the fore, presented a brown trout EC10 of 6.5 

mg/kg in eggs, or around three-fold lower than any other species mean value.

Before accepting the FWS conclusion for brown trout, EPA believed it was necessary to carefully 

examine the issues raised by FWS in its interpretation, because its EC10 is distant from previous 

interpretations of the study and distant from all selenium reproductive EC10s obtained for other species. 

Because Salmo (as previously analyzed by EPA) was already the most sensitive genus in EPA’s internal 
2

draft national criteria document, it had substantial influence on the draft criterion calculated from the 

Species Sensitivity Distribution. Were EPA to accept the FWS conclusion that the EC10 for brown trout 

is 6.5 mg/kg in eggs, the importance of this one study would be magnified relative to studies with all 

other species. A factor of three difference in a national recommended criterion value appears to be at 

stake.

This peer review is thus occurring as a consequence of the importance of the interpretation of this one 

study in derivation of a national aquatic life criterion.

1.3. Post-Peer Review Note (2014)

Following the peer-review of Taulbee et al. (2012) that is embodied in the peer reviewer comments in the

current document, a quality assurance (QA) report of the original raw data from the Formation 

Environmental brown trout reproductive study was conducted (AECOM 2012). In addition to making 

numerous minor corrections to the original data, AECOM (2012) also notably provided counts of fish that 

were fully free from all deformities, as well as the number of fish assessed for deformities. This important 

additional information addressed a major source of uncertainty in the Formation Environmental (2011) 

study raised in the U.S. FWS (2012) review and by peer reviewers commenting on the Taulbee at (2012) 

analysis conducted for EPA, by providing counts for fry that are fully free from all deformities. These 

updated deformity data allowed for an Addendum to the Taulbee et al. 2012 reanalysis that greatly 

reduced the number of hypothetical scenarios for both the deformity and combined (survival plus 

deformity) endpoints that were examined in Taulbee et al. (2012) in order to account for uncertainties in 

reported deformity counts. The reanalysis, included as a 2014 Addendum, addresses many peer reviewers 

comments contained herein. The 2014 Addendum includes optimistic and worst case EC10 calculations 

for survival, deformities (fully normal), and combined (survival + fully normal) endpoints. Because the 

final recommended EC10 in Taulbee et al. (2012) was based on the full (all field-collected samples plus all 

hatchery samples), the six scenarios presented in the addendum were only examined using the full dataset. 

The final recommended EC10 presented in the 2014 Addendum is 15.91 mg Se/kg dw, based on the worstcase scenario for the deformity endpoint, which was the most conservative EC10 among the six scenarios 

that were considered. The value of 15.91 mg Se/kg dw is the chronic value for brown trout recommended 

in the External Peer Review Draft Aquatic Life Ambient Water Quality Criterion for Selenium –

Freshwater 2014 (U.S. EPA 2014). 

1.4. Charge Questions

1. Endpoints. EPA evaluated five endpoints, (a) survival from hatch to swim-up, (b) survival from 

hatch to 15-days post swim-up, (c) deformities, (d) major deformities, and (e) combined survival 

and deformities. Do you agree that these are appropriate endpoints upon which to base an EC10?

2. Examination of uncertainty caused by loss of individuals in aquaria that overflowed, via best 

case and worst case assessment. Concerned that the lost individuals may have been less healthy 

than the individuals not lost, FWS discarded the data available from the overflow aquaria. The 

analysis of EC10s corresponding to 80 different ways to constitute the dataset indicate that the 

FWS survival EC10 and deformity EC10 hinge on removing this evidence. EPA prefers not to 

dismiss potentially useful information unless careful analysis demonstrates that no scientifically 

defensible conclusion can be supported by the information. Do you agree that EPA examined the 

relevant range of reasonable possibilities when it considered that the individuals lost in the 

overflow either (a) had health equivalent to those not lost, or (b) were all dead (dying) or 

deformed (depending on the endpoint being considered) at the time of the event?
3

3. Examination of uncertainty caused by compiling separate counts for each type of deformity. 

ENSR and Formation Environmental report the rates of individual deformities, but not rates of 

combined deformities. To arrive at its EC10, FWS considered one possibility, termed “maximum 

fully normal” for rates of combined deformities. Given the concern that EPA acceptance of a 

“maximum fully normal” calculation would set a potentially underprotective precedent that future 

analysts might follow in such situations (noting that the ENSR-Formation reporting format is not 

unique), do you believe that EPA’s examination of the spectrum of possibilities, from “maximum 

fully normal” (most optimistic) to “minimum fully normal” (worst case), and plus the 

intermediate case (probability based) is scientifically reasonable?

4. Possible use of FWS-calculated EC10 of 6.5 mg/kg in the derivation of a national criterion. 

Using the criteria derivation procedures EPA ordinarily employs, the national criterion for 

selenium can be expected to fall at or below the EC10 for the most sensitive species. Based on 

the 80 different EC10s it presented for brown trout, along with other analyses presented in 

Appendix A, EPA concluded that it would have difficulty defending the validity of the FWS 

EC10 calculation of 6.5 mg/kg, primarily because it hinges on removing the data that indicate a 

much higher EC10, even under worst case assumptions. Considering the material presented in 

the main text and in Appendix A, do you agree with EPA’s concerns about adopting the FWS 

calculation as the critical component in derivation of a national criterion?

5. Use of an EPA-calculated brown trout EC10 of 16.76 mg/kg in a Species Sensitivity 

Distribution for derivation of a national criterion. EPA has classified the brown trout eggs and 

fry used in the study according to their origin, “Field” being from adults wild-caught from the 

study area and “Hatchery” from the two hatcheries. Provided that no Field data are discarded,

EPA recognizes that more than 90 percent of the various EC10s it calculated (including those for 

the comprehensive endpoint “combined survival and deformities”) fall above its recommended 

EC10 of 16.76 mg/kg for brown trout. Nevertheless, EPA believes that it can defend the 16.76 

mg/kg as being a scientifically reasonable, albeit environmentally conservative interpretation of 

brown trout sensitivity, for purposes of deriving a national criterion. Do you agree?

Note A: EPA is not contending that 16.76 mg/kg is the only scientifically appropriate EC10 that 

can be obtained from this study data.

Note B: For purposes of obtaining a study having fewer uncertainties, EPA does not have the 

resources for itself to undertake a new brown trout reproductive study. Nor does it have the 

authority to require anyone else to repeat this study. Although EPA has the option of judging the 

study to be inadequate for use in deriving a national criterion, proceeding with such an option 

would yield a national criterion that is higher than it would otherwise be. More importantly, 

given the FWS assessment, simply rejecting the brown trout study and deriving a national 

criterion without it would leave open a question of whether EPA’s resulting criterion was 

sufficiently protective. (An option of not proceeding with a national criterion in the absence of a 

brown trout EC10 does not appear viable given that brown trout data have never been deemed 

essential, and few criteria incorporate tests with brown trout.) In any case, inherent in Question 5 

is EPA’s belief that given its effort to understand how the study uncertainties affect the resulting 

EC10, the study contains valuable information that should not be discarded. EPA does not 

believe the uncertainties in the brown trout EC10 are much greater than, and may in fact be less 

than those for other studies used in criteria derivation.
4

1.5. Reviewers

In August of 2012 the Charge Questions along with a copy of the draft report Review of draft EPA report, 

‘Analysis of the Brown Trout Selenium Toxicity Study Presented by Formation Environmental and 

Reviewed by U.S. Fish and Wildlife Service’, dated August 22, 2012, was sent to six reviewers and 

included: John Besser, Kevin Brix, David DeForest, Jerome M. Diamond, Anne Fairbrother, and Chris 

Mebane. The sections below are organized by each Charge Question with peer reviewers’ [verbatim] 

comments.

2. CHARGE QUESTIONS, REVIEWERS COMMENTS AND EPA

RESPONSE TO COMMENTS

2.1. Question 1: Endpoints.

EPA evaluated five endpoints, (a) survival from hatch to swim-up, (b) survival 

from hatch to 15-days post swim-up, (c) deformities, (d) major deformities, and 

(e) combined survival and deformities. Do you agree that these are appropriate 

endpoints upon which to base an EC10?

2.1.1.Peer Reviewer 1

Any of these endpoints seem appropriate. There have arguments that reversible deformities such as edema 

and finfold deformities should be treated differently than more severe deformities that will obviously be 

fatal to the individual such as the notorious two-headed fish or other major skeletal deformities 

(McDonald and Chapman 2009; Janz et al. 2010). It was reasonable for EPA to examine both 

“deformities” and “major larval deformities.” However, in the wild, where baseline mortality rates are 

high, any single observable deformity would likely affect the behavior of the fry and lead to death. It is 

interesting that “more conservative” endpoints such as the combined survival and deformities endpoint do 

not necessarily result in lower EC values. The reason for this is usually because if “more conservative” 

endpoints are highly variable in the reference or low exposure treatments, they may not yield as low of an 

EC value as “less conservative” endpoints such as mortality, or may not yield reliable EC values 

whatsoever. I suggest mentioning somewhere in the discussion that “more conservative” endpoints do 

not always yield “more conservative” EC values.

2.1.2.Peer Reviewer 2

An EC10 can be developed using any measured endpoint, therefore, I am interpreting the question to 

mean whether these endpoints are relevant for setting water quality criteria. Within that context, Stephan 

et al. (1985) says that an endpoint should be one that is suitable for the species and life-stage being tested 

(with no comment about chemical-specific specificity). Historically, EPA has considered the ecological 

relevance of an endpoint, specifically whether a stipulated adverse effect can (either theoretically or 

experimentally) be tied directly to changes in fitness parameters (growth, reproduction, or survival). 

Given this context, survival is an obvious endpoint for criteria derivation. Deformities are also 

appropriate if sufficiently severe, as they can affect fish survival and possibly reproductive capacity 

(although the latter has not been directly investigated). What is not known is the severity of deformity 

necessary to inevitably result in an increased probability of decreased survival or productivity of an 

individual. In this study, deformities were measured on live fish, 15-days post swim-up. Presumably, 

therefore, they were not severe enough to result in the death of alevins or very young fry. However, it is 

reasonable to suppose that Level 3 deformities are likely to reduce fitness to some extent during the 

lifecycle of these fish, while Level 1 may not; Level 2 is equivocal in this regard.
5

Therefore, based on the logic described in the above paragraph, I reached the following conclusions:

(a) Survival from hatch to swim-up—This endpoint is appropriate, as survival is a fitness characteristic 

directly related to population persistence. 

(b) Survival from hatch to 15-days post swim-up—Same as above, with one caveat. In this study, the 

post swim-up phase was in clean water with clean food. Therefore, the fry had an opportunity to depurate 

selenium during the 2 weeks that they were on clean food. While any deformities that were initiated prior 

to swim-up are likely to persist, it may be that the survival of these fish is higher than it would be if they 

were in a stream with selenium at the concentrations experienced by their parents. 

(c) Deformities—My preference is to not use this as an endpoint, as it includes Level 1 deformities that 

may or may not be associated with changes in survival or productivity.

(d) Major deformities—Defining these as Level 2 and Level 3 deformities includes a degree of 

conservatism (since we don’t know long-term consequences of Level 2 deformities), but has a high 

likelihood to result in long-term reductions in survival and/or productivity; therefore, I agree with use of 

this endpoint. 

It would be instructive to do some additional exploratory statistics to examine the relationship of 

deformities and selenium concentrations below the pre-threshold concentration (6.2 to 20.5 mg 

selenium/kg egg dw). For example, a step-wise regression analysis beginning with inclusion of only 

Level 3 deformities, then adding in Level 2, and finally Level 1, and examining whether the r2 is robustly 

defined by Level 3 only, might provide insight about whether the lower level deformities are significant.

(e) Combined survival and deformities—This endpoint gives equal weight to deformities and survival. In 

theory, it should result in an EC10 that is lower than the least sensitive endpoint and higher than the most 

sensitive endpoint. As a matter of policy, it seems appropriate for EPA to regulate on the most sensitive 

ecologically-relevant endpoint. Therefore, I recommend that this endpoint not be used and to select 

instead the lowest of the separate survival and major deformities endpoints. Furthermore, I believe that 

these two endpoints (survival and deformities) are addressing different questions. “Survival” is 

addressing whether the alevins are exposed to lethal concentrations (can they survive?) while 

“deformities” is addressing the question of whether survivors will have shortened life-spans or suffer 

productivity consequences later in life from early selenium exposure (albeit only up to 2 weeks post 

swim-up in this study). Therefore, the populations in question (the denominator used to develop the 

proportion of affected individuals) is not the same, so the metrics cannot be combined into a single risk 

estimate. This explains the counterintuitive result of “survival” having a lower EC10 than “deformities” 

(which seems to suggest that all fish die before becoming deformed, which obviously did not happen). 

Note: The explanation in the above paragraph is corrupted to some extent by the fact that for the five 

highest selenium concentrations, very few (or no) fish reached swim-up so preserved alevins were also 

assessed for deformities. This actually provides an argument that not all dead fish are deformed, and adds 

significant uncertainty to the estimate of percent deformities in live fish that survive to 15 days post 

swim-up. 

Although not directly stated, another aspect of this question is whether an EC10 is an appropriate level on 

which to base a criterion. Of course, the selection of a protection level is a policy decision and not a 

scientifically-derived number (although it can be informed by scientific information). However, I think it 

should be noted that an EC10 is lower than what EPA has used historically (EC50 or EC20), thus 

representing an increased level of conservatism in this criterion. One of the arguments against use of an 

EC10 is that the uncertainty bounds are usually large at this part of the concentration-response curve. All 

of the TRAP analyses presented by EPA include confidence limits on the EC10. It would be instructive 

to calculate the EC20 values as well (at least for the primary endpoints/study design selected), to see how 

much overlap there is between the EC10 and EC20 values.
6

2.1.3.Peer Reviewer 3

Of the five endpoints evaluated by EPA, Formation, and FWS, there is no doubt that the survival endpoint 

from hatch to swim-up is valid and useful. This is by far the strongest endpoint from the study because 

this phase of the study was not affected by overflows. 

In reading through Formation’s description of the deformities and how they were scored, I was struck by 

the fact that even a level 1 or level 2 deformity (as Formation described them) appeared to be fairly 

“major” in that a fish having such a deformity would be unlikely to survive and reproduce. For example, 

the Formation Report describes a level 1 for cranio-facial eye deformities as one eye blind or poorly 

developed. This strikes me as fairly major in terms of that fish’s chances of survival. The real issue here 

is what is the baseline rate for these deformities in unexposed wild fish in the region. This issue cannot 

be addressed with the Formation study. In my opinion then, I do not see the need to distinguish “major 

deformities” from “deformities” in general. Furthermore, the summary data presented in the Formation 

report indicates that major deformities constituted a small percentage of the total deformities observed 

(5% or less). Thus, there may not even be sufficient data with which to rigorously analyze differences 

with egg Se concentration, particularly an accurate EC10 estimate. Finally, I would note that other EPA 

toxicity methods that evaluate larval development (e.g., EPA’s echinoderm development test) do not 

distinguish “major” from other deformities. The larva either develop normally or they do not.

The endpoint, survival from hatch to 15-days post swim-up, is obviously fraught with problems because 

of tank overflows and loss of fish in many treatments. This endpoint consumes what I believe is an 

inordinately complicated and tortuous amount of effort in both the FWS Review and in EPA’s reanalysis 

of the data. In examining the survival data in Tables 1a and 1b for those treatments that did not overflow, 

there is virtually no difference in survival at swim-up and 15-days thereafter, despite egg Se 

concentrations as high as 14 mg/kg (e.g., LSV2C012, LSV2C-016 and C350-006). These data suggest 

that there is little reason to believe that survival would be differentially affected during that 15-d period 

and therefore, attempts to make this endpoint usable, given the overflow problem, are probably 

unnecessary. 

The endpoint “combined survival and deformities” is I think unnecessarily complicated and probably not 

useful. What would be more profitable is to examine all of the sublethal endpoints measured, including 

deformities at 15-d post swim-up to see if these yield a more sensitive endpoint than survival at swim-up. 

In this light, I believe EPA’s “Method 2” combined with their “maximum normal” worst case approach, 

addresses the concerns they were attempting to analyze using a combined survival – deformity endpoint.

EPA Question to Peer Reviewer 3: In the final paragraph in response to charge question 1 and in the 

second paragraph in response to charge question 5, you recommend that “EPA consider evaluating 

Method 2 combined with the maximum normally developed approach (defined as “maximum normal 

worst case approach” in response to charge question 1) as the critical worst case condition for the study 

based on the information from the study itself.

First, when you wrote “maximum normal”, was it you intention to write “minimum normal” (maximum 

deformed), which would be the worst case scenario with respect to deformities? In this analysis, 

“minimum normal” was the deformity endpoint with the most conservative assumptions, although in 

practice, EC10s for “maximum normal” were often lower, in large part because of the lack of inadequate 

partial effects, but also because the degree of difference in deformities among hatchery samples and field 

samples was larger in the “maximum normal” scenarios.
7

Peer Reviewer 3 Response: I did not explicitly say so in my review but I found the descriptions for the 

“minimum” and “maximum” normal analyses very confusing and wasn’t at all sure that either was a 

satisfactory option or that either one was “optimistic” or “conservative”. When I originally prepared my 

review I recommended the “minimum normal” based on the text that this was the most conservative 

scenario and I had thought that this scenario was most similar to what I thought was the best option 

which was: “... that one deformity is sufficient to classify a fish as abnormal and therefore not normal”. 

However, when I examined the figures, I observed what EPA noted above; that the EC10s were in fact 

higher for the supposedly most conservative scenario (“minimum normal”) and that the difference in 

deformities between the field samples and hatchery fish was greater under the “maximum normal” 

scenario. When examining Figures 7 and 8 of the Review, the field data in Figure 7 (Review panel) 

appeared to have a more reasonable concentration –response than that shown in Figure 8 using the 

“minimum normal” scenario. I therefore wrote in my review that the “maximum normal” appeared to be 

a more “worst case” scenario than the “minimum normal”. Given the problems with the data in terms of 

calculating reliable EC10s given different subsets of the data (scenarios), perhaps the real answer here is 

to not label any of the scenarios as “conservative” or “worst case” because in fact, they may not be, 

given the EC10 results. If, however, EPA can determine that one of the 3 scenarios examined comes 

closest to the option I noted above: that a fish with even one deformity is sufficient to classify it as 

abnormal and thus affected, then that should be the most conservative option and let the EC10 fall where 

it may.

2.1.4. Peer Reviewer 4

It is well demonstrated in the scientific literature that larval mortality and deformities are the most 

sensitive endpoints in fish exposed to maternally transferred selenium (Se) (Janz et al. 2010). 

Accordingly, the five endpoints evaluated by the EPA are appropriate for deriving an EC10. In my 

opinion, of these five endpoints, the most appropriate endpoint is the combination of mortality plus major 

deformities, as this would provide the most complete measure of adverse effects. Further, this would be 

most consistent with the USEPA’s guidelines for ambient water quality criteria (AWQC) development 

(USEPA 1985), which recommends mortality plus the percentage of test organisms with incompletely 

developed shells as the appropriate endpoint for embryo and larval tests with barnacles, bivalves, and 

other invertebrate species. In other words, organisms with sufficiently severe developmental 

abnormalities would not be expected to survive and reproduce. However, if one of the other endpoints, 

such as survival alone or deformities alone, results in a lower EC10 estimate than the combined survival 

plus deformities endpoint, I believe it is justified to use the lower EC10 estimate as there may be more 

“noise” in the concentration-response relationship for the combined survival plus deformities endpoint 

that could artificially increase the magnitude of the EC10.

2.1.5. Peer Reviewer 5

The endpoints evaluated by EPA are appropriate, and selection of the most sensitive of these endpoints 

for final threshold calculation is also appropriate. My only concern is with the use of the EC10 versus 

EC20. In both the 1999 ammonia criteria document and the draft 2004 Se criteria document, EPA stated 

that the EC20 was the preferred regression-based endpoint. EPA further stated that lower ECx estimates 

(e.g., EC10) were rarely statistically different from the control and had wide confidence intervals, making 

them unsuitable for criteria derivation. The analysis in this report which shows the EC10 is quite 

sensitive to the inclusion or exclusion of certain data, seems to highlight the very reason EPA has 

previously used the EC20. I assume EPA has performed some additional analyses or perhaps made a 

level of protection policy decision on use of the EC10 in the current analysis. Presumably, this issue will 

be addressed in the revised WQC document, but at least a brief indication on why EPA is now focusing 

on the EC10 should be provided in the final version of this report.
8

2.1.6. Peer Reviewer 6

Endpoints (a) and (c) are appropriate and are conventionally reported in maternal-transfer toxicity studies. 

Both endpoints are straightforward and ecologically relevant, and both have been reported as the mostsensitive endpoint in studies with other salmonids. (However, it may also be instructive to estimate 

EC10s for frequencies of individual classes of deformities, which would not require any assumptions 

about associations among deformity types.) The post-swim-up survival endpoint (b) is ecologically 

relevant and it apparently added sensitivity to the test, but it also added increased uncertainty, due to the 

missing larvae. Based on the outcome of the brown trout study, it seems that post-swim up phase is 

somewhat more difficult to conduct (due to the need to provide food), and it may compromise the 

deformity endpoint (by limiting the number of swim-up larvae available for examination). The major 

deformities endpoint (d) seems by definition to be less sensitive and more subjective than (c), but it could 

potentially provide some useful insight into a ‘difficult’ dataset. In most studies, the combined endpoint 

(e) is likely to be dominated by the more sensitive of the two endpoints, and its use in this case seems to 

be a last-ditch effort to clarify ambiguous data.

2.2. Question 2: Examination of uncertainty caused by loss of individuals in aquaria 

that overflowed, via best case and worst case assessment.

Concerned that the lost individuals may have been less healthy than the 

individuals not lost, FWS discarded the data available from the overflow 

aquaria. The analysis of EC10s corresponding to 80 different ways to constitute 

the dataset indicate that the FWS survival EC10 and deformity EC10 hinge on 

removing this evidence. EPA prefers not to dismiss potentially useful 

information unless careful analysis demonstrates that no scientifically defensible 

conclusion can be supported by the information. Do you agree that EPA 

examined the relevant range of reasonable possibilities when it considered that 

the individuals lost in the overflow either (a) had health equivalent to those not 

lost, or (b) were all dead (dying) or deformed (depending on the endpoint being 

considered) at the time of the event?

2.2.1. Peer Reviewer 1

This is a troubling complication of the test. The Formation Report described it as “For some of the test 

chambers, fry were lost during [the post swim-up stage] of the study due to food clogging the drain pipe, 

resulting in an overflow of the test chamber.” Ten of 19 field treatments suffered overflows, and between 

19% and 50% of test organisms per treatment were lost in the overflows.

Arguments can be made for both assumptions tested by EPA, either that the lost fish either had (a) had 

health equivalent to those not lost (Method 1 or M1 in the subject draft report); or (b) were all dead 

(dying) or deformed (Method 2 or M2 in the subject draft report). 

The expected behavior of brown trout swim-up fry would support the Method 1 assumption. Steve 

Brinkman, a toxicologist with the Colorado Division of Wildlife, has conducted numerous early-life stage 

tests with brown trout. He advised that in his experience swim-up brown trout differ from rainbow or 

cutthroat trout in that the browns only fed near the bottom of the aquaria, whereas the rainbows or 

cutthroats will feed at the surface. However, when not feeding, healthy post swim-up brown trout fry are 

not strictly benthic, and will move throughout the water column in the aquarium. Moribund or dead fry 

are not floaters, but sink to the bottom and thus would be unlikely to be lost in an overflow (Steve 

Brinkman, Colorado Division of Wildlife personal communication, August 31, 2012). 
9

From the photo of a test chamber and outflow drain in the Formation Report (repeated at the back of this 

review for convenience), it looks like a fairly coarse mesh covered the drain, and that this gave at least 3 

to 4 cm of freeboard in the tanks. From this admittedly limited view of the tank setup, it looks like it 

might have required a lot of excess food or a several days’ holiday from cleaning the tanks for the drains 

to clog up to that extent. 

My general philosophy is that because toxicological data of the sort generated by the Formation study are 

scarce and difficult to generate, they should never be summarily discarded for nonfatal protocol 

deviations. There are a lot of things that can go wrong in the preparation for and execution of a nonroutine chronic test such as this, and technician fatigue or burnout towards the end of complex tests can 

be a concern. Having a few fish go down the drain doesn’t mean the entire test went down the drain, and 

the careful analyses in the subject EPA report are commendable. Yet this mishap didn’t lose just a few 

fish. To me, losing up to half of the test organisms introduces such uncertainty that the affected treatments 

were potentially compromised, and data interpretations would best focus on the unaffected treatments.

EPA Question to Peer Reviewer 1: Because survival rate in the second part of the test (during which 

some aquaria overflowed) is multiplied by survival rate in the first part of the test, with respect to survival

your page 3 statement is clear to us:

To me, losing up to half of the test organisms introduces such uncertainty that the affected 

treatments were potentially compromised, and data interpretations would best focus on the 

unaffected treatments.

That is, the reason for your preference for discarding the affected treatments from the survival EC10 

calculation is clear to us based on the above statement. However, deformity counts from the two portions 

of the test are merely added together, such that by our count, only 8-10% of the individuals contributing 

to any treatment’s deformity rate were lost in the overflow. Our question is whether you would want to 

clarify your statement of overflow loss relative to a rationale for discarding those treatments from the 

deformity EC10 calculation. We do not want a potential difference of opinion about the best way to 

analyze data to be construed as a difference in the facts (the data themselves) recognized by each of us.

Peer Reviewer 1 Response: I regret not having more time to spend on this review, and cannot take time 

now to go back and try to reproduce how the deformity counts were produced, and why that only 8-10% 

of the individuals contributing to deformity counts were lost. However, I did not find any mistakes or 

calculations that I questioned in any of the counts or statistics in the EPA report, and thus I accept that 

the 8-10% figure was determined appropriately. I did not notice this estimate of 8-10% of the loss of fish 

contributing to deformity counts in my review, and I suggest making this point more explicit in the final. 

Also, I think this is an important enough point and apparent misunderstanding on my part, that I think the 

point should be mentioned in the abstract or summary for the final report. 

I agree that losses of 8 to 10% are not as severe as the 20-50% for the survival endpoints, and thus 

retaining those treatments affected by the overflow loses for the deformity endpoints would seem more 

defensible than would be retaining the greater percent loss for the survival endpoint. Still, the 

explanations for salvaging the data from the overflow-affected treatments are inevitably somewhat 

involved. Further, because the resulting EC10s would be little different whether the affected treatments 

were retained or excluded, ranging only from about 15 to 17 mg/kg dw in my estimates or the estimate 

from the EPA draft analysis, using the a ≈ 15 mg/kg EC10 value from treatments that did not lose fish 

would seem more straightforward, simpler, and thus easier to explain and defend.
10

2.2.2. Peer Reviewer 2

I agree with EPA’s use of the data from the overflow tanks. While my first inclination was to discard that 

information (or rerun the test), the loss of information for the critical range of selenium exposure is a 

major concern, as is the lack of “partial-effect” data for derivation of the EC10 using the TRAP model. 

Without partial effects, there actually is a family of concentration-response curves that can be developed, 

even though the slope between “no effect” and “100% effect” points is essentially unknown. Lack of 

information on the slope is, however, a major drawback as the objective of this test (and other similar 

concentration-response studies) is to define the curve between the “all” or “none” values (or to precisely 

define the threshold between “all” and “some” or “some” and “none”). Therefore, lack of data in the 

intermediate range severely restricts the utility of the study, leading me to the conclusion that the data 

from the overflow tanks should be used, if at all possible. Alternatively, EPA could develop the family of 

curves based on the “all or none” data, and select among them for estimating the EC10. This is not a very 

satisfactory approach as there is no logical basis for selection of one curve (slope) over another.

EPA’s approach of assuming two values for these tanks (proportionality with the other tanks, or all were 

dead or deformed), is appropriate for providing probability bounds around the true value (see, for 

example, Warren-Hicks and Hart 2010). A third assumption might be that all fish in the overflow tank 

survived with no deformities; this would truly bound all potential outcomes. However, such an 

assumption appears highly unlikely, given that there were partial effects in all the tanks that did not 

overflow. Therefore, I agree with EPA’s decision to not make such a calculation. Depending upon the 

degree of conservatism that is desired (again, a policy, not a scientific, decision), either the upper or lower 

bound around the true value could be selected for use in criteria derivation (alternatively, the geometric 

mean of the two could be estimated and used instead).

2.2.3. Peer Reviewer 3

In my opinion EPA examined more than they needed to address the over flow problem. I agree with EPA 

that although discarding the treatments that had overflows is not a good solution in that this will bias the 

EC10 calculation in such a way as to not be representative of the actual effects observed. I also agree 

with both FWS and EPA that treating the missing fish from overflows as if they were never there in the 

first place, is not scientifically defensible. The various scenarios EPA examined cover the possibilities 

and perhaps this was necessary in order to clearly see the results of “worst case” assumptions in terms of 

the EC10. However, my thinking would be to focus on the truly worst case, that all missing fish were 

deformed or dying (i.e., EPA’s Method 2) and that one deformity is sufficient to classify a fish as 

abnormal and therefore not normal (i.e., EPA’s “maximum fully normal” approach). This approach I 

think best tests whether deformities is a more sensitive endpoint than survival at swim-up.

2.2.4. Peer Reviewer 4

In my opinion, the EPA clearly and quantitatively evaluated the full range of reasonable possibilities for 

evaluating the uncertainties associated with the fish lost to aquaria overflow. See my response to Charge 

Question #4 for additional thoughts on this issue and implications for calculating EC10s.

2.2.5. Peer Reviewer 5

The EPA evaluation of this issue is quite rigorous. From this evaluation, it is clear that the FWS 

threshold only occurs when data are removed leading to an unreliable estimate of the EC10 (see comment 

above regarding EC10 versus EC20). The scenario in which all overflow fish were assumed to be Se 

impacted is particularly convincing that the true EC10 is 16-17 mg/kg. The USFWS threshold of 6.5 

mg/kg is not scientifically supportable given the EPA analysis of this issue.
11

2.2.6. Peer Reviewer 6

The two scenarios selected to estimate the status of larvae missing due to overflows are relevant and 

reasonable. They may represent a slight bias toward more adverse outcomes, in that the worst-case 

scenarios probably overestimated adverse effects more frequently than the optimistic scenarios 

underestimated them. However, such a bias is appropriate in this case, because it is more conservative 

and because the alternative would be to discard potentially important data.

2.3. Question 3: Examination of uncertainty caused by compiling separate counts for 

each type of deformity.

ENSR and Formation Environmental report the rates of individual deformities, 

but not rates of combined deformities. To arrive at its EC10, FWS considered 

one possibility, termed “maximum fully normal” for rates of combined 

deformities. Given the concern that EPA acceptance of a “maximum fully 

normal” calculation would set a potentially underprotective precedent that 

future analysts might follow in such situations (noting that the ENSR-Formation 

reporting format is not unique), do you believe that EPA’s examination of the 

spectrum of possibilities, from “maximum fully normal” (most optimistic) to 

“minimum fully normal” (worst case), and plus the intermediate case 

(probability based) is scientifically reasonable?

2.3.1. Peer Reviewer 1

It seems to me that unless there are non-statistical reasons for preferring a certain endpoint or method of 

calculating an endpoint, evaluating the spectrum of possibilities as was done in the present report seems 

commendable and appropriate. By “non-statistical” I mean if there were reasons to invoke based on 

ecotoxicology or practical reasons like reproducibility of deformity measurements, or whether some 

deformities are more likely attributable to selenium than other factors. These sorts of factors support 

looking at the “Free from Major Deformities” endpoint separately, from total deformities.

This again leads to the point, that in this approach to data analysis, the optimistic or pessimistic 

assumptions about deformities do not a priori lead to higher or lower EC values; it depends on the 

distribution of responses in the reference (or in the case of this study where there was no reference, the 

low range of exposure data).

2.3.2. Peer Reviewer 2

Use of probability bounds to define the range of possible (or likely) outcomes is a well-supported 

statistical approach to dealing with uncertainty (Warren-Hicks and Hart 2010). Therefore, in concept, 

EPA’s approach to defining possible upper and lower bound (best case, worst case) analyses is 

appropriate. EPA’s approach to calculating the intermediate case (assuming independence of deformities)

is a third reasonable approach that should result in a mid-range value. 

These ranges of possible proportion of deformities were calculated for all deformities considered together, 

and also calculated separately for major deformities (Level 2 and 3). My preference is to not group all 

deformities together, but rather to do the calculations for Level 1 separately, and then group Level 2 and 3 

for a second calculation. The probability of having a minor deformity (Level 1) appears to be greater than 

having a major deformity (Level 2 or 3), therefore potentially skewing the data if they are considered 

concurrently. As discussed in a previous question (No. 1), it is likely that Level 3 and possibly Level 2 

deformities have ecological relevance, while it is unlikely (or at least, much less likely) for Level 1, 

supporting the argument that these two groups should not be combined.
12

2.3.3. Peer Reviewer 3

I found this to be the most difficult and complicated aspect of the studies reviewed, including EPA’s 

analysis. After examining the information, I am less and less convinced that all of the approaches 

evaluated be EPA were really necessary. In particular, I think that the “intermediate fully normal” 

approach, while interesting, is not necessarily well-founded, nor is the relationship between number of 

deformities and “deformity rate” well understood at this point. In my opinion, this approach is 

speculation that I think is unnecessary in terms of extracting usable information from the Formation 

study. Again, I think the analysis may be overly complicated by trying to evaluate deformity rate and its 

multiple expressions depending on how one treats the various types and levels of deformities. In addition, 

while the Formation report describes various quality assurance/quality control (QC) procedures to 

categorize deformities, the literature indicates several issues in characterizing and quantifying deformities 

as a toxicity endpoint in selenium studies (e.g., McDonald and Chapman, 2009; Rudolph et al, 2007). 

McDonald and Chapman (2009) discuss possible issues with the edema deformity category, for example 

and they identify several internal and external QC checks that do not appear as if they were done in the 

Formation study. Finally, while the graduated severity index (GSI) developed by Holms et al (2005) and 

used in the Formation study has merit, Rudolph et al (2007) and deBruyn et al (2007) published what I 

believe is a better definition for each GSI level that, if used would help to distinguish major from minor 

deformities. Between the overflow problems, the inherent variability in deformity types and levels, and 

the lack of true controls in this study with respect to deformity rate, I believe this analysis should be kept 

simple, conservative, and very transparent.

2.3.4. Peer Reviewer 4

It is unfortunate that the combined rate of total deformities was not reported, but, as EPA noted, this is 

common in several of the Se toxicity studies that have been published. However, I am aware of at least 

three studies in which rates of individual deformities and total deformities in Se-exposed fish are both 

reported: Holm (2002), Muscatello et al. (2006), and McDonald et al. (2010).

 Holm (2002) provides deformity data for rainbow trout (Oncorhynchus mykiss) larvae reared 

from parent fish collected from a Se-exposed site and from a reference site. Deformities were 

reported both as total deformities, as well as individual deformities (craniofacial, skeletal, 

finfold, edema). The data for the Se-exposed site are summarized in Table 1. As these data 

show, the “maximum fully normal” and “intermediate fully normal” assumptions would 

result in estimated total normal larvae percentages that bracket the measured percentage of 

total normal larvae in all three of the samples, while the “minimum fully normal” assumption 

would underestimate the percentage of total normal larvae (Table 2).

 Muscatello et al. (2006) evaluated Se-induced deformities in northern pike (Esox lucius) 

larvae reared from parent fish collected from a reference site and three Se-exposed sites with 

low, moderate, and high Se concentrations. Like Holm (2002), deformities were reported 

both as total deformities, as well as individual deformities (craniofacial, skeletal, finfold, 

edema) (Table 3). As shown in Table 2, measured percentages of total normal larvae are the 

same, or very close to the same, as those estimated using the “maximum fully normal” 

approach when the rate of deformities is low (less than approximately 12% deformities). 

Overall, the “maximum fully normal” approach more closely predicts the percentage of 

normal larvae than does the “intermediate fully normal” approach.

 McDonald et al. (2010) evaluated Se-induced deformities in Dolly Varden (Salvelinus 

malma) larvae reared from parent fish collected from a Se-exposed site and a reference site. 

Deformity data from this study, based on moderate and severe deformities (i.e., a graduated 

severity index (GSI) score of 2 or 3), are summarized in Table 4. As shown in Table 2, the 

measured percentages of total normal larvae are the same, or very close to the same, as those 
13

estimated using the “maximum fully normal” approach when the rate of deformities is low 

(less than approximately 10% deformities). 

In order to synthesize the data from the three studies, Figure 1 provides a plot of the measured rates of 

total normal larvae versus estimated rates of total normal larvae based on the “maximum fully normal,” 

“intermediate fully normal,” and “minimum fully normal” approaches. As shown, when the deformity 

rates are less than approximately 10% (i.e., total normal larvae rates of greater than 90%), either of the 

three “fully normal” approaches provides a reasonable estimate of the measured percentages of normal 

larvae. However, as the rate of deformities increases, the “maximum fully normal” rate tends to 

underestimate the rate of total deformities, while the “minimum fully normal” and “intermediate fully 

normal” approaches start to greatly overestimate the rate of total deformities. The R2values based on the 

linear relationships between the estimated total normal larvae and measured total normal larvae, with the 

intercept set equal to 0%, are 0.94, 0.83, and 0.64 for the maximum, intermediate, and minimum 

approaches, respectively. Accordingly, the correlation between the estimated and measured percentages 

of total normal larvae is best for the “maximum fully normal” approach. However, the “maximum fully 

normal” approach, on average, over-predicts the total normal larvae rate by approximately 8%, while the 

“intermediate fully normal” approach under-predicts the total normal larvae rate by approximately 15%. 

Focusing on just those treatments where the measured rate of total deformities was <10%, which are of 

particular importance in terms of deriving an EC10, the “maximum fully normal” approach over-predicts 

total normal larvae by just 1% and the “intermediate fully normal” approach under-predicts total normal 

larvae by approximately 5%. An additional possibility for estimating the rate of total normal brown trout 

larvae would be to use the relationship between the “maximum fully normal” data, which provided the 

closest estimates of total normal larvae, and measured total normal data from Table 2. Based on the 

resulting linear regression equation shown in Figure 2, the estimated rates of fully normal brown trout 

larvae tend to fall between the “maximum fully normal” and “intermediate fully normal” rates presented 

in Taulbee et al. (2012). 

Based on the above analysis, it appears that the best estimate of fully normal larvae may be somewhere 

between the maximum and intermediate approaches. Given this, I recommend consideration of both 

approaches in the weight-of-evidence evaluation. Caution should still be used when evaluating this 

approach for complete concentration-response relationships, as the “intermediate fully normal” approach 

is likely to over-predict larval deformities at higher egg Se concentrations (at least given the analysis of 

the data from Holm [2002], Muscatello et al. [2006], and McDonald et al. [2010]). The influence of these 

higher predicted deformity rates on the curve used to derive the EC10 should be evaluated to ensure that 

they do not artificially pull the EC10 down below egg Se concentrations associated with low rates of 

deformities. 

Table 1. Percentages of total deformities, craniofacial deformities, skeletal deformities, finfold 

deformities, and edema in rainbow trout (Oncorhynchus mykiss) larvae collected from a Seexposed site. Data estimated from Figure 2.6 in Holm (2002).

Sample

Egg Se 

(mg/kg 

wet wt.)1

Egg Se 

(mg/kg 

dry wt.)2

Total 

Deformities 

(%)

Craniofacial 

Deformities 

(%)

Skeletal 

Deformities 

(%)

Finfold 

Deformities 

(%)

Edema 

(%)

1 6.8 17.4 7.0 1.4 4.5 0.7 1.7

2 6.9 17.7 20 7.3 13 1.6 5.0

3 11.5 29.5 91 15 24 7.3 86

1 Holm (2002) reported wet weight concentrations.

2 Dry weight concentrations estimated assuming a moisture content of 61% (Holm et al. 2005).
14

Table 2. Comparison of “maximum fully normal”, “intermediate fully normal”, and “minimum 

fully normal” larvae based on individual deformities to the measured percentage of total 

normal larvae.

Study

Egg Se 

(mg/kg dry 

wt.)

Maximum 

Fully Normal

(%)

Intermediate 

Fully Normal 

(%)

Minimum 

Fully Normal

(%)

Measured 

Total Normal 

(%)

Holm (2002) 17.4 96.5 91.9 91.7 93

17.7 87.0 75.4 73.1 80

29.5 14.0 8.4 0 9

Muscatello et al. (2006) 3.19 94.3 84.6 83.7 91.4

3.19 89.8 85.5 85 90.9

3.19 88.8 73.6 70.7 85.8

3.19 91.7 81.6 80.4 91.8

48.23 68.5 34.9 9 55.2

48.23 63.9 31.5 2.1 64.7

31.28 77.5 48.4 34.6 65.9

31.28 80.8 50.4 37.9 70.3

3.80 88.2 69.4 65.5 84.9

3.80 88.5 75.4 73 87.9

McDonald et al. (2010) 51.9 99.1 99.1 99.1 99.1

56.3 97.9 91.9 91.6 97.9

56.6 93 81.7 80.4 91.5

60.5 83.5 70.8 67.9 80

65.8 70.4 48.2 35.9 64.2

5.4 94.2 94.2 94.2 94.2

Table 3. Percentages of total deformities, craniofacial deformities, skeletal deformities, finfold 

deformities, and edema in northern pike (Esox lucius) larvae collected from a reference site and 

three Se-exposed sites with low, moderate, and high Se concentrations. Data estimated from

Figures 1 and 3 from Muscatello et al. (2006).

Sample

Egg Se 

(mg/kg dry 

wt.)

Total 

Deformities 

(%)

Craniofacial 

Deformities 

(%)

Skeletal 

Deformities 

(%)

Finfold 

Deformities 

(%)

Edema 

(%)

RF/RW 3.19 8.6 4.5 5.7 1 5.1

RF/HW 3.19 9.1 2.5 10.2 0 2.3

RF/MW 3.19 14.2 9.0 11.2 2.3 6.8

RF/HW 3.19 8.2 5.6 8.3 0 5.7

HF/HW 48.23 44.8 31.5 24.3 10.3 24.9

HF/RW 48.23 35.3 36.1 26.8 9.0 26.0

MF/MW 31.28 34.1 22.5 18.5 5.2 19.2

MF/RW 31.28 29.7 19.2 18.5 5.2 19.2

LF/LW 3.80 15.1 11.8 11.5 1 10.2

LF/RW 3.80 12.1 8.5 11.5 1.3 5.7

RF = reference site fish; HF = high Se site fish; MF = medium Se site fish; LF = low Se site fish; RW = 

reference site water; HW = high Se site water; MW = medium Se site water; LW = low Se site water
15

Table 4. Percentages of total deformities, craniofacial deformities, skeletal deformities, finfold 

deformities, and edema in Dolly Varden (Salvelinus malma) larvae collected from a Se-exposed 

site. Data from Table 2 in McDonald et al. (2010). Data only shown for treatments in which 

deformities were observed with a GSI score of 2 or 3 (i.e., moderate or severe).

Sample

Egg Se 

(mg/kg dry 

wt.)

Total 

Deformities 

(%)

Craniofacial 

Deformities 

(%)

Skeletal 

Deformities 

(%)

Finfold 

Deformities 

(%)

Edema 

(%)

1 51.9 0.9 0 0 0 0.9

2 56.3 2.1 2.1 2.1 2.1 2.1

3 56.6 8.5 2.8 7 5.6 4.2

4 60.5 20 0.9 16.5 13 1.7

5 65.8 35.8 8.6 29.6 22.2 3.7

6 5.4 5.8 0 0 0 5.8

Figure 1. Relationship between the measured rate of total normal larvae versus the estimated 

rate of total normal larvae based on the “maximum fully normal”, “intermediate fully normal”, 

and “minimal fully normal” approaches. Deformity data from Holm (2002), Muscatello et al. 

(2006), and McDonald et al. (2010); see Table 2.

0

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 60 70 80 90 100 Estimated Total Normal (%)

Measured Total Normal (%)

Maximum fully normal approach

Intermediate fully normal

Minimum fully normal approach

1:1 agreement between measured 

and estimated total deformities
16

Figure 2. Relationship between the measured rate of total normal larvae versus the estimated 

rate of total normal larvae based on the “maximum fully normal” approach. Deformity data 

from Holm (2002), Muscatello et al. (2006), and McDonald et al. (2010); see Table 2.

2.3.5. Peer Reviewer 5

I agree that the maximum fully normal rates are potentially underprotective and that EPA was correct to 

evaluate the other scenarios in their review. Note, it is also possible that the minimum fully normal 

approach is overprotective. From a mechanistic point of view, I think it unlikely that the different types 

of deformities are fully independent of each other and so the minimum fully normal scenario may be 

mechanistically unrealistic. It is interesting that the most conservative scenario (minimum fully normal) 

generally resulted in the highest EC10s. Looking at the dose response relationships for the different 

scenarios, it seems this may be the result of the high response in the control using this scenario. Given 

this unexpected result, for future studies, it would be prudent to continue evaluating all three scenarios, 

selecting the most conservative result. Obviously, the most preferred scenario is one in which data are 

collected and presented in a manner that indicates whether different types of observed deformities are cooccurring in the same fish. In this report and in any subsequent WQC document, EPA should explicitly 

make this recommendation so researchers will design their data collection appropriately in the future. 

2.3.6. Peer Reviewer 6

EPA examined the widest possible range in frequencies of combined deformities. The inclusion of the 

worst-case scenario may be justifiable for the sake of completeness, but the underlying assumption (that 

different types of deformities have a strong negative association) is counterintuitive and contrary to the 

trends observed in previous studies. The ‘intermediate case’ is more realistic but still represents a weaker 

y = 1.0759x - 9.8812

R2 = 0.9634

0

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 60 70 80 90 100 Measured Total Normal Larvae (%)

"Maximum Fully Normal" Larvae (%)
17

association among deformities (i.e., no association) than would be expected. The ‘maximum normal’ 

scenario is the most optimistic, but to some extent it represents the more realistic assumption that 

different deformities are positively associated.

2.4. Question 4: Possible use of FWS-calculated EC10 of 6.5 mg/kg in the derivation 

of a national criterion.

Using the criteria derivation procedures EPA ordinarily employs, the national 

criterion for selenium can be expected to fall at or below the EC10 for the most 

sensitive species. Based on the 80 different EC10s it presented for brown trout, 

along with other analyses presented in Appendix A, EPA concluded that it 

would have difficulty defending the validity of the FWS EC10 calculation of 6.5 

mg/kg, primarily because it hinges on removing the data that indicate a much 

higher EC10, even under worst case assumptions. Considering the material 

presented in the main text and in Appendix A, do you agree with EPA’s 

concerns about adopting the FWS calculation as the critical component in 

derivation of a national criterion?

2.4.1. Peer Reviewer 1

I do not believe that an EC10 of 6.5 mg/kg is a reasonable most sensitive EC10 estimate from this study, 

although my reasons differ from those in the subject EPA analysis of these data. The charge question (and 

much of the EPA analysis) is concerned with the removal of data from treatments that the in the FWS 

review were considered compromised. While I eventually came to the conclusion that simply excluding 

the treatments which were affected by the overflow mishap would be the most straightforward and 

defensible approach, I think the 6.5 mg/kg value is an artifact of the curve fitting model used. My best 

estimates of an EC10 from the same data were around 13 to 15 mg/kg Egg Se (dw). 

I was able to nearly reproduce the 6.5 mg/kg EC10 from the Review data set. The plots of my 

reproduction and those from Figure 4 of the EPA analysis are nearly identical, as is the corresponding plot 

in the FWS Review (their Figure 2). The data points and curve fits appear almost identical (my figures, 

Figure 1A and 1B). An inspection of the plots shows five important features: (1) there is no 

concentration-response in the field data up to at least 13 mg/kg (about 1.1 Log(Egg Se mg/kg) units, the 

scale used in Fig 1); (2) the four points in the plot at about 1.1 Log(Egg Se mg/kg) units form a line that 

visually define the boundary of the no apparent effect region of the data, a “no effect line”; (3), the noeffect inflection point in the curve fit occurred well below (i.e., to the left of) the “no-effect line” (4) there 

was a good fit to the Spring Creek Hatchery control (SPC) data points clustered near the vertical axis at 

0.73 mg/kg Egg Se; and (5) the highest concentrations were censored (excluded from the curve fit) which 

improved the fit with the SPC low Se treatments and also tugged at the location of the curve inflection.
18

L og(Egg Se, mg/kg dw )

Maximum Ful ly N ormal Fry

.2 .4 .6 .8 1.0 1.1 1.2 1.4 1.6 1.8

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1.0

1.1

L og(Egg Se, mg/kg dw )

Maximum Ful ly N ormal Fry

.2 .4 .6 .8 1.0 1.1 1.2 1.4 1.6 1.8

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1.0

1.1

L og(Egg Se, mg/kg dw )

Maximum Ful ly N ormal Fry

.2 .4 .6 .8 1.0 1.1 1.2 1.4 1.6 1.8

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1.0

1.1

A. Closest attempt to reproduce the 6.5 mg/kg EC10 

from the “Review” dataset (field data with no missing 

fry+Spring Cr. Hatchery control (SPC)); triangular model 

and censoring the two highest treatments (open 

circles).

P ECp (mg/kg

10 6.3

0 2.8

C. Curve fits for Maximum Fully Normal fry from the 

“Review” dataset; triangular model, no censoring

L og(Egg Se, mg/kg dw )

Max. N ormal

.2 .4 .6 .8 1.0 1.1 1.2 1.4 1.6 1.8

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1.0

1.1

P ECp (mg/kg)

10 13.0

0 8.9

D. Same as “A” except using piecewise linear fit

P ECp (mg/kg)

10 13.1

0 11.5

E. Same as “D” except no censoring

P EC (mg/kg dw)

10 13.5

0 12.2

B. Original curve fit that produced the 6.5 mg/kg 

EC10 from the “Review” dataset (field data with no 

missing fry+Spring Cr. Hatchery control (SPC)). 

Figure was copied from Fig. 4 of the EPA analysis.

Figure 1. Curve fit that nearly reproduced the 6.5 

mg/kg EC10 from Charge Question #4, plus 

alternative fits with the same “FWS Review” 

dataset.

There were no effects apparent in these 

4 treatments all at about 13 mg/kg Egg 

Se. Thus the EC10 estimate should not 

be less than the 13 mg/kg “no-effect 

line” from these 4 points. Yet, the no 

effect point estimate from EC curve was 

2.8 mg/kg.

P ECp (mg/kg

10 6.55

0 Not reported

“no-effect line”
19

To reproduce the curve fit that matched the fit to all the datapoints in Figure 1B and the EC10 of 6.5, I 

had to censor (exclude) the highest Se treatments with no survival. Because the curve fit models 

supported by TRAP are all require symmetrical, the severe effect datapoints will tug at the curve shape in 

the low effects region of the data and will also affect low-effect EC estimates (Scholze et al. 2001; 

Erickson 2010). Because low effect ECs (e.g.. EC0s, EC10s, EC20s) are what are of concern, not high 

effect ECs (e.g., EC80s, EC100s), censoring high effect datapoints to improve the curve fits for the low 

ECs is an appropriate work around to the symmetry limitations of TRAP. However, there is no mention of 

censoring either in the EPA analysis or the USFWS Review, so maybe there were other differences in the 

data used or analyses. (Thus the recommendation that the final EPA report clearly label which datapoints 

went into each EC10 estimate, and any pretreatments, etc.). Regardless, the point of the exercise was to 

reproduce the fit, and because the no-effect curve inflection, or EC0, occurred well below the apparent 

“no-effect line” in Figure 1B, this is not an ideal curve fit to these data.

Figure 1C shows the same Review dataset fitted with the same Tolerance Distribution‒Triangular 

Distribution model but without censoring the high effect points, and Figures 1D and 1E use a rectangular 

(uniform) distribution, which gives a “hockey stick” or “broken stick” shaped model. The math for this 

latter model is different from that used in the more familiar piecewise linear regression, although the 

model shape and EC values between the two are nearly identical. All of these alternatives produce EC0s 

and EC10s between 9 and 14 mg/kg that bracket the “no effect line” whereas the curve fit in Figure 1B 

produced an EC0 and EC10 of 2.8 and 6.3 mg/kg which were well below the no effect line. Therefore, I 

would not use the 6.5 mg/kg EC10 estimate from this study as a most sensitive endpoint summary EC10 

from the study for use in species sensitivity distribution (SSD) analyses. This conclusion is also consistent 

with the view of peer reviewer #2 of the FWS Review (David Janz), that in his opinion the most 

appropriate value was “likely somewhere between the 6.5 μg Se/g egg dw EC10 value proposed by the 

FWS reviewer and the 17.7 μg Se/g egg dw EC10 value proposed in the report” (USFWS 2012, at p. 58 

of 154). 

It is worth noting that the question whether an EC10 of 6.5 mg/kg egg selenium is appropriate from the 

Formation study is distinct from the question whether it is plausible that a species could be that sensitive. 

The technical charge letter mentions that in the data available to Janz et al (2010), the EC10s for eight fish 

species from five families all few within a narrow band of 17-24 mg/kg dw egg Se. As the diversity of 

data grows, this band will likely widen. For example, reduced egg production in Desert pupfish has been 

reported with egg selenium concentrations as low as 4.4 mg/kg dw (Besser et al. 2012).

EPA Questions to Peer Reviewer 1: We are having difficulty recreating your recommended EC10s, 

particularly with regard to your recommended EC10s using tolerance distribution modeling. The 

information you provided in Table 1 of your review are percentages, which can only be used in nonlinear 

regression modeling. Could you please provide the data used to perform the tolerance distribution 

modeling shown in Figures 2 and 3, or provide Table numbers and columns from the EPA report where 

the information was acquired.

Peer Reviewer 1 Response: I entered the percentages as the number of organisms without response, and 

entered 100 as the total number of organisms. Because the tolerance distribution approach changes the 

data to proportions, I believe doing this shortcut provides the same or nearly the same results as entering 

the straight numbers of organisms exposed. This shortcut was a convenience because data in the EPA 

report were only provided as PDF pages. I attached a zip file with the various specific results and the 

spreadsheet for entering data into TRAP.
20

2.4.2. Peer Reviewer 2

EPA’s concern is consistent with the guidance provided by Stephan et al. (1985) who state: “The 

agreement of the data within and between species should be considered. Acute values that appear to be 

questionable in comparison with other acute and chronic data for the same species and for other species in 

the same genus probably should not be used in calculation of a Species Mean Acute Value.” 

Furthermore, there is precedence to taking a weight-of-evidence approach where the preponderance of the 

scientific evidence is weighed when reaching a legal or policy decision (Krimsky 2005). Therefore, 

outlier data should be viewed with suspicion unless or until confirmed with additional evidence, 

particularly in the absence of a plausible hypothesis to explain their occurrence.

2.4.3. Peer Reviewer 3

I completely agree with EPA that the 6.5 mg/kg EC10 derived by FWS is not supported by the data with 

or without the overflow problem. Clearly, a minor difficulty with the Formation Study as noted 

previously is the lack of a true wild trout control treatment from this region with egg Se concentrations 

similar to hatchery fish eggs. However, the Formation study does demonstrate that treatments 

representing less impacted conditions and having higher egg Se concentrations (and that did not overflow: 

e.g., CC350-006, CC150-015, and CC-150-009) had excellent survival throughout the study and 

deformity rates similar to hatchery fish. Thus, the FWS EC10 is not supported by the data.

2.4.4. Peer Reviewer 4

In my opinion, the most compelling case for not discarding the data from the “overflow” aquaria is that 

egg Se EC10s as low as 6.6-8.6 mg/kg dry wt. are only obtained by excluding all data from “overflow” 

aquaria, while egg Se EC10s across all other scenarios are >15.81 mg/kg dry wt. even in scenarios where 

it is assumed that all of the missing fry were either dead or abnormal. This strongly suggests that the basis 

for the lower EC10s of 6.6-8.6 mg/kg dry wt. is strictly a statistical artifact resulting from the elimination 

of intermediate Se treatments and has no toxicological relevance to Se. 

Aside from the finding that egg Se EC10s <15.81 mg/kg dry wt. could not be derived even when 

assuming that missing fish from the “overflow” aquaria were 100% dead or abnormal, there does not 

appear to be a strong basis for making the assumption that all missing fish were dead or abnormal. The 

aquaria that had an overflow issue were all associated with the intermediate cluster of egg Se 

concentrations that ranged from 6.2-20.5 mg/kg dry wt. This cluster of intermediate egg Se concentrations 

comprised 19 aquaria, of which 10 had an overflow issue. The mean egg Se concentration of 10.3 mg/kg 

dry wt. in the “no overflow” aquaria was not significantly different than the mean egg Se concentration of 

11.4 mg/kg dry wt. in the “overflow” aquaria (Student’s t-test; p = 0.563). Likewise, there were no 

significant differences in percent larval survival in the “overflow” and “no overflow” aquaria when 

missing fish are excluded (Mann-Whitney test; p = 0.583), nor in the estimate of “intermediate fully 

normal larvae” (Student’s t-test; p = 0.279). The lack of significant differences in larval survival and 

deformities between the “overflow” and “no overflow” aquaria, when missing fish are excluded, suggests 

that the fish were not lost to the overflow because they were dead or deformed. If the missing fry were 

lost to the overflow because they had died, a comparable reduction in survival would be expected in the 

“no overflow” aquaria (if the missing fry had in fact died prior to the overflow event and if the mortalities 

were due to Se and not some other source). However, this was not the case. As Figure 2 in Taulbee et al. 

(2012) nicely shows, survival in the “no overflow” aquaria ranged from approximately 93-99%, while 

survival in the “overflow” aquaria would range from just 48-82% if it was assumed that all missing fish 

had died prior to the overflow event. Given the similar ranges in egg Se concentrations between the 

“overflow” and “no overflow” aquaria within the intermediate cluster of egg Se data, including 

statistically indistinguishable mean egg Se concentrations, the likelihood that there was an increased 

mortality rate in only those aquaria that happened to have an overflow issue is extremely unlikely. This 
21

issue would be more complicated to interpret if all of the “overflow” aquaria were associated with egg Se 

concentrations that were higher than in the “no overflow” aquaria. In my opinion, there is no basis for 

assuming that all missing fish were either dead or abnormal, and all statistical analyses for determination 

of EC10 values should be based on exclusion of missing fish from the “overflow” aquaria.

The overall weight-of-evidence does not appear to support an egg Se EC10 of 6.5 mg/kg dry wt. First, 

even if it were assumed that all missing fish were dead or deformed, the lowest EC10 that can be derived 

from all other scenarios is 15.81 mg/kg dry wt. Second, based on my reasoning above, I do not believe 

there is justification for excluding the “overflow” data from the concentration-response analysis. Given

this, along with my opinion to only consider the “maximum fully normal” and “intermediate fully 

normal” approaches for estimating the percentage of total normal larvae, the lowest reasonable EC10 

reported in Taulbee et al. (2012) would be 16.58 mg/kg dry wt. Accordingly, even the most conservative 

EC10 from what I consider to be the reasonably relevant range of data analysis scenarios, is 

approximately 2.5 times greater than the EC10 of 6.5 mg/kg dry wt.

2.4.5. Peer Reviewer 5

As discussed above, use of the FWS EC10 does not appear to be reasonable. Inclusion of the treatments 

excluded by FWS with the assumption that all lost fish were Se impacted clearly demonstrates that the 

FWS EC10 is driven by a poorly characterized dose response. As pointed out in the report, the dose 

response used to derive the FWS EC10 is uncharacteristically shallow for Se and appears to be largely the 

result of excluding the treatments in which fish were lost to overflow. Further, all other species tested 

(including 3 other salmonids) have very similar EC10s (17-24 mg/kg), again suggesting the FWS EC10 is 

anomalous. 

2.4.6. Peer Reviewer 6

It is not reasonable to use the FWS EC10 for deformities for derivation of a national (or site-specific) 

criterion for selenium. Calculation of this EC10 was dependent on several questionable assumptions, as 

was illustrated by the EPA’s alternative analyses (all based on the same ‘maximum normal’ estimate for 

deformities):

 FWS considered some fish that did not reach swim-up stage to be ‘fully normal’. Although some 

of these alevins did not have any visible deformities, trout that fail to swim up can reasonably be 

considered ‘not fully normal’. (All alevins that failed to swim up were considered to be 

mortalities in the FWS survival analysis.) If all larvae that failed to swim up are classified as ‘not 

fully normal’, the EC10 increases to 13.0 ug/g. 

 FWS used control data from the SC hatchery, but not the SPC hatchery. The SPC samples were 

the best method control because they had the same rearing protocol as field-collected samples, 

whereas the SC controls had a different rearing protocol. If the SPC data are included in the 

analysis instead of the SC data, the EC10 increases to 13.7 ug/g. 

 FWS excluded data from any samples with missing larvae, ignoring the information on 

deformities in the larvae remaining in these samples. EPA’s analysis demonstrated that field 

samples with missing larvae did contain reliable data that helped to define the concentrationresponse curve. The low FWS EC10 value is inconsistent with the absence of any negative trend 

in deformities at egg-Se concentrations below 20.5 ug/g in the complete dataset (see Figure 7, 

‘All’). If field samples with missing larvae are included -- and all missing larvae are assumed to 

be deformed -- EC10s ranged from 18.4 ug/g to 19.2 ug/g, depending on which control group(s) 

are included. 
22

2.5. Question 5: Use of an EPA-calculated brown trout EC10 of 16.76 mg/kg in a 

Species Sensitivity Distribution for derivation of a national criterion.

EPA has classified the brown trout eggs and fry used in the study according to 

their origin, “Field” being from adults wild-caught from the study area and 

“Hatchery” from the two hatcheries. Provided that no Field data are discarded, 

EPA recognizes that more than 90 percent of the various EC10s it calculated 

(including those for the comprehensive endpoint “combined survival and 

deformities”) fall above its recommended EC10 of 16.76 mg/kg for brown trout. 

Nevertheless, EPA believes that it can defend the 16.76 mg/kg as being a 

scientifically reasonable, albeit environmentally conservative interpretation of 

brown trout sensitivity, for purposes of deriving a national criterion. Do you 

agree?

Note A: EPA is not contending that 16.76 mg/kg is the only scientifically 

appropriate EC10 that can be obtained from this study data.

Note B: For purposes of obtaining a study having fewer uncertainties, EPA does 

not have the resources for itself to undertake a new brown trout reproductive 

study. Nor does it have the authority to require anyone else to repeat this study. 

Although EPA has the option of judging the study to be inadequate for use in 

deriving a national criterion, proceeding with such an option would yield a 

national criterion that is higher than it would otherwise be. More importantly, 

given the FWS assessment, simply rejecting the brown trout study and deriving 

a national criterion without it would leave open a question of whether EPA’s 

resulting criterion was sufficiently protective. (An option of not proceeding with 

a national criterion in the absence of a brown trout EC10 does not appear viable 

given that brown trout data have never been deemed essential, and few criteria 

incorporate tests with brown trout.) In any case, inherent in Question 5 is 

EPA’s belief that given its effort to understand how the study uncertainties 

affect the resulting EC10, the study contains valuable information that should 

not be discarded. EPA does not believe the uncertainties in the brown trout 

EC10 are much greater than, and may in fact be less than those for other studies 

used in criteria derivation.

2.5.1.Peer Reviewer 1

From my earlier suggestion to Charge Question #2 that the most straightforward and defensible treatment 

of the test data is to consider the treatments with missing fish potentially compromised, it follows that the 

lowest test EC10 from the non-compromised dataset would be most appropriate to use for calculating 

species and genus mean chronic values for a species sensitivity distribution. I recommend considering this 

change for internal consistency, although as illustrated below, as a practical matter, the EC10 values 

calculated from the set of field and Saratoga hatchery control treatments which not lose fry will not be 

greatly different from the 16.76 mg/kg value recommended in the draft EPA analysis.

Response curve fits for the “Maximum Fully Normal Fry” endpoint and “Full Test Survival” (hatch to 

test termination 15 days post swim-up) are shown in Figures 2 and 3 respectively. The data used were 

limited to the field treatments the Saratoga Hatchery treatments that did not lose fish during the tank 

overflow. The Saratoga Hatchery treatments are used in lieu of the Spring Creek Hatchery fish because 
23

the Saratoga fish were obtained at the same time as the field collected fish, and the eggs and embryos 

were similarly handled. 

The Spring Creek Hatchery brown trout embryos are probably not as good of comparisons to the field 

collected brown trout as are the Saratoga Hatchery fish because the Spring Creek fish were obtained as 

eyed embryos rather than gametes, and were held in the lab for a shorter period (see also the peer review 

appendices to the USFWS (2012) review by M. Rigby, D. Janz, and V. Palace). While the fungal 

problems complicate interpretation of the Saratoga hatchery results, the field collected brown trout also 

shared some of these complications. It seems plausible to me that fungal infestations could cause 

deformities in early-life stage fish, since deformities in frogs have been linked to fungal exposures and 

pathogenic infections or parasites can cause deformities in older fish, such as whirling disease. However, 

I found little support for this notion from some cursory literature searches that I made in Scopus and 

Google Scholar, as well as general internet searches. Regardless, if the fungal infestation had caused 

deformities, it presumably would have affected both the Saratoga and Field data sets, and an inspection of 

the data plots in Figure 2 will show that the distribution of deformities in the Saratoga Hatchery 

treatments are similar to those from the Field collected brown trout, up to the curve breaks at about 

Log(Egg Se) values of 1.1. The Saratoga samples are plotted with the same symbols from the Field Data, 

but can be distinguished because they are bunched at the low range of selenium exposures, <0.2 Log(Egg 

Se) in the plots in Figure 2. In contrast, the Spring Creek Hatchery treatments shown in Figure 1 have a 

lower variability and overall higher proportions of fully normal fry. There are two interpretations that 

could be made from this: 1) most of the field collected brown trout with selenium >0.8 Log(Egg Se), i.e., 

greater than about 6 mg/kg Egg shown in Figure 1 have elevated fry deformities; or 2) the Spring Creek 

Hatchery fish represent a different population of data that should not be pooled with the field collected 

data. I lean toward the latter interpretation, and have excluded the Spring Creek Hatchery fish from the 

statistical comparisons. 

Plots and EC estimates from the Field and Saratoga datasets of the Maximum Fully Normal Fry, using 

only those “non-compromised” treatments which did not lose fish in the overflow mishap, show that 

EC10 values are around 15 mg/kg Egg Se, regardless if these Field data were analyzed with or without 

the Saratoga data, or if a triangular model or piecewise linear regression is used. 
24

L og(Egg Se, mg/kg dw )

Maximum Ful ly N ormal Fry

.2 .4 .6 .8 1.0 1.2 1.4 1.6 1.8

0

10

20

30

40

50

60

70

80

90

100

110

L og(Egg Se, mg/kg dw )

Maximum Ful ly N ormal Fry

.2 .4 .6 .8 1.0 1.1 1.2 1.4 1.6 1.8

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1.0

1.1

L og(Egg Se, mg/kg dw )

Maximum Ful ly N ormal Fry

.2 .4 .6 .8 1.0 1.1 1.2 1.4 1.6 1.8

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1.0

1.1

L og(Egg Se, mg/kg dw )

Maximum Ful ly N ormal Fry

.2 .4 .6 .8 1.0 1.2 1.4 1.6 1.8

0

10

20

30

40

50

60

70

80

90

100

110

A. Recommended non-compromised dataset to consider 

relying upon: Maximum fully normal fry, from field and 

Saratoga (SC) treatments which had no missing fry, tolerance 

distribution, triangular model

B. Same dataset as “A” but nonlinear regression, piecewise 

linear model shape

B. Same analysis as “A” but only using field data (the SC data, 

shown in open circles, are shown but were not used in the curve 

fit)

D. Same analysis as “B” but only using field data (the SC data, 

shown in open circles, are shown but were not used in the curve 

fit)

Figure 2. Curve fits using only the field and Saratoga (SC) treatments which did not lose fry, for the 

Maximum fully normal fry endpoint

L og(Egg Se, mg/kg dw )

Maximum Ful ly N ormal Fry

-.2 0 .2 .4 .6 .8 1.0 1.2 1.4 1.6 1.8

0

10

20

30

40

50

60

70

80

90

100

110

E. Same analysis as “A” but setting 3 data points for Log(Egg Se) 

of 1.4 to 1.6 and which had partial responses to zero F. Same analysis as “A” but setting 3 data points for Log(Egg 

Se) of 1.4 to 1.6 and which had partial responses to zero 

L og(Egg Se, mg/kg dw )

Maximum Ful ly N ormal Fry

-.2 0 .2 .4 .6 .8 1.0 1.2 1.4 1.6 1.8

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1.0

1.1

P ECp (mg/kg)

10 13.7

0 12.7

P ECp (mg/kg)

10 15.5

0 14.6

(Egg Se, mg/kg)

P ECp LCL UCL

10 14.9 12.7 17.6

0 10.7 8.4 13.6

(Egg Se, mg/kg)

P ECp LCL UCL

10 14.3 13.0 15.8

0 12.8 11.5 14.2

(Egg Se, mg/kg)

P ECp LCL UCL

10 15.6 12.8 19.0

0 11.3 8.4 15.2

(Egg Se, mg/kg)

P ECp LCL UCL

10 14.7 9.0 23.8

0 13.4 7.7 23.2
25

The curve fits for Maximum Fully Normal Fry endpoint in Figures 1 and 2 benefit from three high Se 

treatments in which partial effects resulted from the 1.4 to 1.6 Log(Egg Se) treatments, that is they had 

lower percentages of normal fry than the scatter of points for the treatments with lower Egg Se 

concentrations. The subject EPA reanalysis notes that it could be considered paradoxical to consider any 

of the fish in these treatments free of deformities, since none of the fish in these three treatments reached 

swim-up by the end of the test. This developmental delay could be considered a deformity in itself, and so 

all of these fish were considered deformed, or at the least, none could be considered fully normal. 

Exploring the implications of that reasoning (top of p. 63 and Figure A-1 of the subject EPA analysis), I 

repeated some of the curve fittings after setting the “Maximum fully normal rates” for these treatments to 

zero. This change and assumption had negligible effects on the ECp values (Figures 2E and 2F). 

However, it does illustrate a caution with relying on the same model for curve fitting in data sets with 

inadequate partial effects to get good EC values. Datasets with inadequate partial effects are vulnerable to 

large interpolation errors since the curve inflection points and response shape are unknown. While TRAP 

has the flexibility to still provide EC estimates in these instances (along with appropriate cautions), these 

estimates are little better than a graphical estimate. The curve shape, steepness, and EC values are thus 

influenced by the model shape chosen. In Figure 2E, the triangular model shape forces an unnatural 

steepness to the response with the curve going to zero just above 1.2 Log(Egg Se) units where the actual 

data don’t go to zero until 1.4 Log(Egg Se) units. This problem is also apparent in some of the plots in the 

EPA analysis. The piecewise linear regression curve is free from this artifact (Figure 2F). Instead of 

simply using the tolerance distribution, triangular distribution model for all the curve fits, I suggest using 

piecewise linear regression for datasets with inadequate partial effects.

Setting the Maximum Fully Normal Fry endpoint to zero for the fish that did not achieve swim-up 

conflates the deformity and survival endpoints into another combined endpoint. The logical extension of 

the approach might be to apply a stunted growth threshold to early life stage tests with fish, where fish 

that fall X% behind reference fish in growth are considered too stunted to survive in the wild and their 

survival or growth was set to 0%. Granted, fish that don’t reach swim-up cannot survive, so it is 

completely appropriate to consider them dead for the survival analysis. I do not see much benefit from 

doing so for the deformity data. I suggest dropping this point from the EPA analysis of deformity data, 

and keeping the partial responses for these three treatments as is, without setting them to zero for the 

Maximum Fully Normal Endpoint.
26

(Egg Se, mg/kg dw )

Survival, 15d Post sw imup fry

1 10 15 20 25 30 35 40 45

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1.0

1.1

L og(Egg Se, mg/kg dw )

L arval survival, ful l test

-.2 0 .2 .4 .6 .8 1.0 1.2 1.4 1.6 1.8

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1.0

1.1

L og(Egg Se, mg/kg dw )

L arval survival, ful l test

-.2 0 .2 .4 .6 .8 1.0 1.2 1.4 1.6 1.8

0

10

20

30

40

50

60

70

80

90

100

110

A. Recommended non-compromised dataset 

to consider relying upon: Survival data from 

hatch to 15 days post swim-up (“full test”), 

from field and Saratoga (SC) treatments 

which had no missing fry, nonlinear 

regression, piecewise linear model

Figure 3. Curve fits using only the non-compromised dataset: Full test survival, from the field and Saratoga 

(SC) treatments which had no missing fry,.

L og(Egg Se, mg/kg dw )

L arval survival, ful l test

-.2 0 .2 .4 .6 .8 1.0 1.2 1.4 1.6 1.8

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1.0

1.1

(Egg Se, mg/kg)

P ECp LCL UCL

10 15.1 14.4 15.9

0 11.8 10.7 12.8

B. Similar to “A” but using the tolerance 

distribution with a triangular distribution

C. Similar to “B” but only noncompromised field data, triangular 

distribution, log transformation

D. Similar to “C” but only non-compromised 

field data, triangular distribution, but no 

transformation since transformation is not 

helpful with the limited exposure range.

(Egg Se, mg/kg)

P ECp LCL UCL

10 14.9 14.1 15.8

0 14.0 13.1 14.9

(Egg Se, mg/kg)

P ECp LCL UCL

10 15.0 cannot be calculated

0 13.8

(Egg Se, mg/kg)

P ECp LCL UCL

10 14.8 142 15.4

0 12.4 11.7 13.2
27

This takes us to the Full Test Survival data, which is the focus of Charge Question #5. When the survival 

data are likewise limited to the Field or Field + Saratoga treatments that did not lose fish during the tank 

overflow mishap, the EC10 values are all very close to 15 mg/kg Egg selenium (Figure 3). Thus the 

survival EC10s are essentially the same as the Maximum Fully Normal Fry endpoints. Because the Full 

Test survival endpoint can be explained in a criteria document with a lot fewer words than are needed for 

the Maximum Fully Normal Fry endpoint, and perhaps with debate less debate, I suggest taking the Full 

Test survival endpoint forward for use in a criterion SSD. I would limit the datasets to the Field and 

Saratoga sets that did not lose fish in the overflow mishap (Figure 3).

The charge questions also lead to a secondary question, ‘what does it mean to be conservative’? The point 

that the 16.76 mg/kg EC10 was lower than 90 percent of the EC values calculated seems immaterial. 

Chronic toxicity tests typically produce a plethora of endpoints to interpret and it seems customary that 

the most sensitive, test endpoint is carried forward as an overall test value for calculating species mean 

chronic values and species sensitivity rankings. So I suppose using an EC10 value from this test that is 

lower that 90 percent of the possible EC values that could be calculated could be considered to be a 

conservative approach, but it also has been the usual approach. It certainly seems appropriate to me, but I 

am not sure the “albeit conservative” caveat is needed.

Note B to Charge Question #5 makes the point that brown trout data have never been considered essential 

for criteria development in the USA. That view seems quite reasonable since the natural distribution of 

brown trout apparently never extended west of Iceland, although brown trout have been introduced and 

subsequently become widely distributed in North America (Elliott 1989). While generally valued and 

regulated by state game and fish departments as a game fish, brown trout have sometimes been 

considered invasive and a competitor of native fishes including redband trout (i.e., native inland rainbow 

trout), cutthroat trout, brook trout, and sculpin (Wang and White 1994; Waters 1999; Zimmerman and 

Vondracek 2005, 2007).

The final point in Note B to Charge Question #5 is “EPA’s belief that given its effort to understand how 

the study uncertainties affect the resulting EC10, the study contains valuable information that should not 

be discarded. EPA does not believe the uncertainties in the brown trout EC10 are much greater than, and 

may in fact be less than, those for other studies used in criteria derivation.” I concur with this view, and 

note that this study has been vetted to a degree that I have never before encountered. This study has 

numerous strengths, however limitations include the overflow mishap and the lack of treatments collected 

from the wild, from regionally low selenium reference sites. The lack of samples from reference sites is an 

issue that has been detailed in the USFWS review, including its appended peer reviews which are 

important reviews in themselves. However, if it is assumed that the Saratoga Hatchery fish are a good 

surrogate for field collected fish from reference sites, the further analyses I show in Figures 2 and 3 

suggest that the lack of fish from a true field reference site would have little influence on the EC10. 

However, this is an assumption, and if instead reference fish had egg selenium concentrations in the 2 or 

3 mg/kg dw, and if the occurrence of deformities in those fish were more like the Spring Creek Hatchery 

fish, then a curve and EC10 similar to that in Figure 1A would result. That the untested true reference 

fish would behave more like the Spring Creek Hatchery fish than the Saratoga Hatchery Fish seems 

improbable to me, but in the absence of data on this question, a conservative approach to evaluating the 

remaining data would seem appropriate to me.

Thank you for the opportunity to review this report and please let me know if additional information is 

required. Please note that while I give my professional contact information, I completed this review on 

my own time, and with nominal exceptions for communications, without encroaching on government or 

other project funding.
28

2.5.2. Peer Reviewer 2

I disagree with EPA’s choice of 16.76 mg selenium/kg egg dw as the correct interpretation of brown trout 

sensitivity. This value is based on the survival, using EPA’s Method 2 (worst case estimate) endpoint. 

This method assumes that all fish lost from the overflow tanks or other types of technician error did not 

survive. The rationale for this is that brown trout alevins generally sit at the bottom of the tank, and those 

that are near-death would be unable to maintain this position, rise higher in the tank, and therefore have a 

higher probability of being lost in the overflow. However, this is a very conservative approach that is 

based on qualitative judgment, and results in a value that is below the concentration at which there was 

100% mortality (measured, not estimated). Therefore, it is my opinion that it is more appropriate to 

assume that the fish lost to overflow have similar ratios of dead/alive fish as those that remained behind. 

Assuming that fish lost to overflow would have survived in the same proportion as those in the tanks that 

did not overflow, results in an EC10 survival estimate of 20.48 mg selenium/kg egg dw (CI: unavailable 

due to lack of “partial kill” data). One the one hand, this is not a very satisfactory result due to the 

difficulty of generating a reliable dose-response relationship but on the other hand it is more satisfying in 

that it fits the available data more closely (as EPA noted, no fish died below this value). 

Making this assumption results in a survival-based EC10 that is higher than the EC10 based on major 

larval deformities, which is also a more intellectually satisfying result. This is based on the EC10 

calculated with an “intermediate number” of normal individuals, regardless of whether missing fry are 

assumed to all be abnormal or were simply not included. The values for these two assumptions are: 

16.70 mg selenium/kg egg dw (CI: 15.88–17.56) or 16.95 (CI: 16.11–17.84) mg selenium/kg egg dw, 

respectively. Both are based on TRAP models free of errors. Interestingly, these values are essentially 

the same as EPA’s survival-based value of 16.76 mg selenium/kg egg dw, but this is likely just an artifact 

of the data and not biologically based.

Note that the values for the deformity EC10s used above assume that the “intermediate FMD ” category is 

the correct analysis to use when estimating percent deformities. I believe this is a conservative, but 

acceptable estimate for the following reasons. The “minimum FMD” summed the probability of each 

deformity to calculate the proportion of fish with one or more deformities. This approach assumes that 

each deformity is mutually exclusive; that is, if a fish has one type of deformity it can’t have any other. 

We know that this is not true, so this “minimum FMD” case should not be used. The second approach 

estimated the “intermediate FMD” (the probability of the joint occurrence of all four deformities) by 

multiplying the rates of each deformity; this assumes the occurrence of each is independent of the other. 

This is more plausible, but still unlikely as all the deformities were based on similar initiating 

mechanisms (e.g., selenium interference with protein formation in embryos). The third approach assumes 

that the maximum number of fish that would be deformed is the maximum for any one deformity, which 

very likely underestimates the total deformed fish as suggested by EPA. The correct manner for 

determining the actual number of deformities is to use a conditional probability, that is, to determine the 

probability that a fish would have one deformity given that they had another. There would be four 

factorial (4! = 24) possible combinations for which probabilities would need to be calculated. This is 

difficult and the information is not available to do so. Therefore, among the three choices presented by 

EPA, the one with the least amount of uncertainty (and with some conservatism) is the “intermediate 

FMD.”

My major reservation is the inclusion of hatchery fish in the calculation. The study was designed to set a 

site-specific criterion for the Smokey Canyon Mine area, and the hatchery fish were included as part of 

the laboratory quality assurance program. In order for the test to be considered valid, hatchery fish had to 

meet minimum survival and maximum deformity rate acceptability criteria, thereby potentially biasing 

the low end of the curve. Therefore, the site-specific concentration-response curve should be defined 

only by fish from the area of interest. However, as concluded by EPA, the study is insensitive to whether

or not the hatchery fish are included as the EC10s with or without them are virtually the same for the 

endpoints selected. 
29

EPA should not simply reject the brown trout study and derive a national criterion without it. Despite the 

problems associated with this test (eyed eggs from SPC hatchery; fungal infections; overflow tanks), there 

is sufficient information to interpret the data, examine probability bounds (“worst case, best case”), and 

reach a reasonable conclusion about the appropriate endpoint and which value to use in the species 

sensitivity distribution (SSD) calculations. Whether the final value selected is based on the survival 

endpoint as suggested by EPA, or the deformities endpoint argued above, will make no difference in the 

SSD calculations as the EC10 values are virtually identical.

2.5.3. Peer Reviewer 3

The EC10 that should be used for the Formation Study depends on several factors in my opinion, each of 

which could be scientifically defended. Factors I consider important in terms of the EC10 for this study 

are: (a) what is truly believed to be the worst case condition based on what occurred in the study, not 

based on the resulting EC10 estimates of different scenarios; (b) the degree to which the two key 

treatments (LSV2C-017 and LSV2C-008) are weighted in the analysis and how they are treated; and (c) 

the importance of model fit and errors for a given EC10.

In terms of what occurred in the study, as discussed in my response to (2) above, I believe that treating 

missing fry as deformed or poorly developed (and therefor affected) is a conservative assumption, which 

may be founded in reality as trout tend to be at the bottom of a tank if possible. I would recommend that 

EPA consider evaluating Methods 2 combined with the maximum normally developed approach as the 

critical “worst case” condition for this study based on the information from the study itself. As noted 

above, EC10 calculations hinge on two data points: LSV26-008 (17.7 mg/kg egg Se) and LSV2C-017 

(20.5 mg.kg egg Se), both of which unfortunately experienced overflows and lost fish during the post 

swim-up phase. These data points are critical because there were no other data points between 15 and 25 

mg/kg egg Se in this study and this is precisely the range in which there is a steep gradient in effects on 

trout. Most of the 80 analytical variations EPA evaluated are of little avail in correcting this problem. It 

is a problem inherent in the test design. Because there are really no partial responses in this range (except 

by making certain assumptions regarding these two treatments in terms of deformity rate), most EC10 

estimates will be subject to errors. However, as EPA points out, the range of possibilities is small; i.e., 

the EC10 is somewhere between around 15 and 20 mg/kg egg Se.

The EC10 calculation is slightly affected by the SPC hatchery data as well. This treatment should not be 

included in my opinion because these eggs were treated very differently from any of the other eggs. I do 

think they serve as a good QA/QC for the study overall but their data should not be used in calculations. I 

therefore would recommend examining the “field” plus the SC control treatments in EC10 analysis. As 

noted in my response to (1), I would not recommend EC10s based on either “major” deformities alone or 

the combined survival-deformity endpoint.

It is clear in reviewing the EC10s calculated based on deformities that most EC10s are greater than the 

EC10s based on survival over the entire test, particularly based on Method 2. This is largely due to the 

variability observed in deformity rate with egg Se concentration as EPA notes. Unfortunately the 

Formation Study did not comprehensively characterize egg Se concentrations and deformity effects using 

good field controls and a range of egg concentrations in the critical region of effects. Egg selenium 

concentrations are presented as a point for each treatment and it was not clear whether there could be a 

range of Se concentrations in eggs for a given treatment. If there was a range of Se concentrations that 

might be an uncontrolled factor in trying to relate deformities to egg Se concentration. 

Given the apparent variability observed in deformities in the Formation Study, I believe that EPA’s 

Method 2 is a reasonable worst-case approach that not only addresses the overflow problem but also 

essentially incorporates the theoretically more sensitive deformity endpoint. Method 2 assumes that all 

missing fish were equivalent to being dead (i.e., weak, deformed), which is a worst case condition given 

that some of the fish lost to the overflow could have been normal. Based on the results of Table 9b, I 
30

would concur with EPA in using the EC10 = 16.76 based on Method 2 and all data as this endpoint 

appears to approximate a no effect threshold exhibited in the data. The fact that there were no error 

messages with the EC10 in this case is a plus although it is not clear to me that this is a major issue for 

most of the scenarios evaluated.

EPA Question to Peer Reviewer 3: Does your recommended endpoint differ from the scenarios we 

presented where we examined the worst case scenario for deformities “minimum normal” fry combined 

with the assumption that all fry lost to overflow were dead or deformed? EC10s for this endpoint were 

presented in Table 10 under the subheading “Missing fry included as abnormal” as the Minimum Normal 

calculation method for the “Wild”, “Review+Overflow”, and “All” data subsets. Associated figures are 

shown in Figure 8. If you did in fact mean “maximum normal” fry as the worst case scenario, because 

even though it had the least conservative assumptions, it often resulted in the lowest EC10 values, then 

those EC10s can also be found on Table 10, as the Maximum Normal calculation method, and are shown 

in Figure 9. For deformities, the assumption “missing fry included as abnormal” was analogous to 

“Method 2” for survival.

Peer Reviewer 3 Response: I was not intending to suggest a new endpoint but rather to recommend using 

“Method 2” (i.e., missing fish were deformed) along with the option that a fish having even one type of 

deformity was abnormal and therefore affected. I did not consider it important to the analysis of this 

study as to whether to “...assume that the less prevalent types of deformities only occur in individuals 

already having the most prevalent type (“maximum normal”) or whether to “...assume that no fish can 

have more than one type of deformity (“minimum normal”). I found both of these options much too 

complicated given the available data. If, however, EPA can demonstrate that such an analysis was 

necessary because of the available data and that a more simplistic conservative analysis was either not 

feasible or unsupported given the data, then I would recommend EPA use what they considered to be 

biologically the most conservative approach of the 3 scenarios examined. 

EPA Question to Peer Reviewer 3: Do you recommend a particular data subset for the endpoint you 

recommend?

Peer Reviewer 3 Response: No, I cannot recommend a particular data subset without doing a much more 

extensive review of the raw data, none of which was available for this review.

2.5.4. Peer Reviewer 4

Based on my review of the brown trout Se study and the analyses presented in Taulbee et al. (2012), my 

opinion is that (1) data for the “overflow” aquaria should not be excluded; (2) there is no reason to 

assume that the fish lost to the overflow were dead or deformed (similar mortality and deformity rates 

would have been expected in the “no overflow” aquaria which had similar egg Se concentrations); (3) 

total normal larvae estimates probably fall somewhere in between that predicted by the “maximum fully 

normal” and “intermediate fully normal” approaches; and (4) the most comprehensive endpoint is 

combined larval mortality and deformities. Based on the above opinions, the relevant egg Se EC10s 

reported in Taulbee et al. (2012) would range from 20.70-21.60 mg/kg dry wt. (Table 5). One would 

expect the combined larval mortalities and deformities endpoint to be more conservative than the larval 

mortalities or larval deformities endpoints alone. However, variability in the concentration-response data 

for the combined endpoint results in a less conservative EC10 than that obtained by some of the methods 

for evaluating larval survival or larval deformities separately. If the endpoints based on larval survival 
31

alone or larval deformities alone are thus considered, the relevant EC10s from Taulbee et al. (2012) range 

from 16.58-21.76 mg/kg dry wt., with a geometric mean of 19.81 mg/kg (Table 2). Overall, therefore, the 

EPA’s recommended EC10 of 16.76 mg/kg dry wt. for brown trout is on the lower end of what I think to 

be the range of possible EC10s from this study. As such, I agree that it is a scientifically reasonable and 

environmentally conservative EC10 for derivation of AWQC.

Table 5. Summary of egg Se EC10s from Taulbee et al. (2012) based on my opinion that (1) data 

for the “overflow” aquaria should not be excluded; (2) fish lost to the overflow should be 

excluded from the analysis; and (3) total deformities should be estimated using both the 

“maximum fully normal” and “intermediate fully normal” approaches. 

Endpoint Data Subset

Egg Se EC10 

(mg/kg dry wt.)

Larval survival, full test Field 20.48

Field + SPC 20.47

All 20.48

Larval survival, hatch to swim up Field 20.72

Field + SPC 20.64

All 20.71

Larval deformities Field 18.86-20.13a

Field + SPC 19.78-21.20a

All 19.73-21.32a

Major larval deformities Field 16.63-18.07a

Field + SPC 16.58-17.96a

All 16.95-18.40a

Larval mortalities and deformities Field 21.60-21.76a

Field + SPC 20.94-21.39a

All 20.70-21.42a

Arithmetic mean: 19.87

Geometric mean: 19.81

a Range reflects total normal larvae estimates based on the fully and intermediate normal approaches.

2.5.5. Peer Reviewer 5

I generally agree with EPA on its opinion regarding the selection of the 16.76 mg/kg EC10 and the 

retention of this study for criteria derivation. Given the uncertainty in this study, it is appropriate to take a 

conservative approach in selecting the EC10. I think an argument could be made for setting the threshold 

at 16.42 mg/kg which is the lowest of the non-FWS data set analyses for which there were no error 

messages in TRAP (missing fry abnormal; intermediate FMD; field+SPC), but this will have a trivial 

impact on criteria derivation and there are concerns about how to best interpret the deformity data as EPA 

has pointed out. Given the uncertainties in the data, I would strongly oppose selecting an EC10 higher 

than 16.76 mg/kg for this study.

Regarding the EPA’s conclusion that uncertainties in this study are equal to or less than in other studies 

used in criteria derivation, I think ultimately this is true. The loss of fish during the overflow event I 

would not consider a standard level of uncertainty for studies used in criteria derivation. However, the 

analysis demonstrating that assuming all overflow fish were Se-impacted results in an EC10 comparable 

to those derived for other species effectively eliminates this potentially significant uncertainty in the 
32

study. If calculated EC10s under this scenario were significantly lower, the study would be much more 

problematic to interpret.

2.5.6. Peer Reviewer 6

I agree that the EPA-recommended EC10 for survival is a reasonable and environmentally conservative 

representation of the brown trout study. EPA selected their recommended EC10 of 16.76 ug/g for larval 

survival based on analyses of data from all samples in the study (including samples with missing larvae) 

with two assumptions: (1) failure to swim up was equivalent to larval mortality; and (2) all missing larvae 

were all dead and deformed. The EPA-recommended value was the lowest EC10 estimate for survival 

except those estimated from the Review dataset (Table 9a). It is clear from the plot of this concentrationresponse curve (Figure 15, ‘All’) that the EPA EC10 is defined primarily by the worst-case estimates of 

survival of missing larvae, especially the two samples with egg-Se of 17.7 ug/g and 20.5 ug/g. These two 

data points clearly indicate that the FWS EC10 of 14.5 ug/g is too low, and the absence of a downward 

trend in either the definitive (no missing larvae) or estimated survival data at egg-Se of 20.5 ug/g or less 

also suggests that the EPA EC10 is conservative and may slightly underestimate the true survival EC10 

for brown trout. 

The EPA concluded that the deformities endpoints were less sensitive than survival, based on the weight 

of evidence from many analyses of data on deformities, major deformities, and survival plus deformities. 

Of all the analyses of the deformities endpoints using the full dataset, only the EC10 for all deformities 

(maximum fully normal) using the ‘intermediate case’ estimates (15.8 ug/g; Figure 9) was slightly lower 

than the survival EC10. However, the confidence interval for this EC10 completely overlapped the 

confidence interval for the survival EC10, reflecting the high variability of the deformities data across a 

wide range of egg-Se-concentrations. Most noticeably, this deformities EC10 was substantially less than 

the two samples with higher egg-Se (17.7-20.5 ug/g), which had background levels of deformities. The 

plot of these data (Figure 9, ‘All’) is consistent with EPA’s suggestion that the true EC10 for deformities 

may be greater than 20.5 ug/g.

3. GENERAL COMMENTS BY REVIEWERS OTHER THAN RESPONSE 

TO CHARGE QUESTIONS

3.1. Peer Reviewer 1

This is a review of the subject U.S. Environmental Protection Agency (EPA) draft report titled “Analysis 

of the Brown Trout Selenium Toxicity Study Presented by Formation Environmental and Reviewed by 

U.S. Fish and Wildlife Service” (Taulbee et al. 2012). The EPA report consists of both reanalyses of 

original data reported by Formation Environmental (2011), also referred to as the “Formation Report” for 

shorthand, and also EPA’s review of a previous review by the U.S. Fish and Wildlife Service of the same 

original toxicity tests with brown trout and selenium (USFWS 2012), also referred to as the “USFWS 

Review” for shorthand.. The USFWS report in turn appended detailed technical peer reviews which are 

important reviews in and of themselves. 

In addition to the specific “charge questions” that follow, I offer a few general observations. Firstly, is 

recognition that although the derivation of aquatic life criteria under EPA’s National Guidelines is 

intended to be a highly structured process, many decisions such as those with which the present review 

are concerned with are inevitably judgment calls. As the preface to the Guidelines note, they

“undoubtedly do not cover all situations that might occur in the future. All necessary decisions should be 
33

based on a thorough knowledge of aquatic toxicology and an understanding of these Guidelines and 

should be consistent with the spirit of these Guidelines, i.e., to make best use of the available data to 

derive the most appropriate criteria.” (Stephan et al. 1985). The last phrase seems to particularly pertain 

to the present report which is a careful effort to make best use of scarce data.

Secondly, this was a dense read, and some editorial improvements would make the revised version of the 

present report more accessible. The number of new terms and abbreviations for the multiple scenarios 

analyzed is almost overwhelming to follow. For the final version, more consistent use of the same 

abbreviations throughout would help. Likewise, if tables and figures could stand alone with all nonstandard abbreviations spelled out in the footers, it would be far easier to follow than having to flip 

around to find the first usage of an abbreviation. For instance, deformity levels 1, 2, and 3 are discussed 

but never defined. The reader shouldn’t have to go back to an earlier document to figure out what those 

are. While the data used to make effect concentration (EC) estimates are presented in detail, I had a hard 

time determining which of the data in Tables 1-8 were actually used in the tables of EC10 values (Tables 

9-12) and I was not able to reproduce some EC10s. Perhaps also add a “Test Number” column to the data 

tables with a simple 1 to 36 designation. This would make it easier to follow which data subset is which. 

The present designation style with LSV2C-0XX might roll off the authors’ tongues, but it was hard to 

keep track of it for those not so deeply involved. The review required a lot of flipping back and forth 

between the curve fitting plots and tables of EC values. Overlaying a summary of the EC values on each 

plot along the lines of the following figures would be easier to follow, as would internal hyperlinks/cross 

references for reading onscreen. A summary or abstract would also be helpful. 

3.2. Peer Reviewer 3

Before responding to the charge questions, I would note that in any other circumstances I would 

recommend that EPA not use the Formation Study (or Fish and Wildlife Service (FWS) Review) because: 

(a) there was no appropriate control in this study; wild caught fish with whole body and egg selenium 

(Se) concentrations similar to the hatchery fish (i.e., < 1.5 mg/kg Se) used in this study would have been 

desirable at the least. As I explain in my responses below, hatchery fish used in this study yield little 

usable information in my opinion; (b) with or without laboratory errors in this study, the fact is that there 

were only two data points out of 26 that had egg Se concentrations in the range already known to be 

critical in terms of potential effects on early life stage fish survival and development (15-20 mg/kg Se), 

and both of these treatments happened to be among those with serious laboratory errors. Thus, the entire 

analysis of this study hinges to a large extent on two data points and; (c) laboratory errors regarding the 

overflow of almost a third of the treatments presents serious challenges in terms of evaluating post swimup effects of Se using these data (as both the FWS Review and EPA acknowledge). 

However, I fully understand EPA’s desire to finalize a Se criterion using the best science and the best data 

available. In this light, I agree with EPA that there are usable data in the Formation study and that EPA 

should attempt to make the best use of this study as they develop a Se criterion.

3.3. Peer Reviewer 5

General Comments: Overall, I found the EPA analysis of the Formation Environmental brown trout 

study and subsequent report by FWS to be very rigorous and thoughtful. I am largely in agreement with 

the analysis, interpretation and conclusions as presented by EPA and have only a few minor comments in 

response to the questions below.
34

3.4. Peer Reviewer 6

I organized my comments into three parts: Part A discusses several problems that could be considered 

serious enough to compromise the scientific validity of the test, making it unsuitable for use in derivation 

of national water quality criteria. Part B discusses the appropriateness of EPA’s alternative scenarios 

used to examine uncertainties in the dataset. Part C addresses the five specific queries presented in the 

Charge for Reviewers.

[Note: I refer to the original report on the brown trout selenium toxicity study by Formation 

Environmental as the ‘Report’, I refer to the review by FWS as the ‘Review’ or ‘FWS’, and to the 

draft EPA document as the ‘EPA analysis’ or ‘EPA’.]

A. FACTORS AFFECTING THE VALIDITY OF THE TEST

Lack of field-collected reference samples

The Report did not include field-collected samples (fertilized eggs) of brown trout from any 

appropriate reference sites. Measured egg-Se concentrations from fish collected at the two upstream 

Crow Creek sites (6-14 ug/g egg-Se; all concentrations dry weight basis) apparently reflect elevated 

levels of Se in fish tissues, presumably due to movement of fish upstream from more contaminated 

stream reaches (Sage Creek or lower Crow Creeks). I agree with FWS and EPA that these levels are 

above expected background concentrations. Efforts to collect ripe brown trout at other presumed 

uncontaminated sites outside the Sage/Crow Creek watersheds were apparently unsuccessful.

The lack of reference samples had little or no effect on characterization of the larval survival 

endpoint, because little or no mortality was observed at egg-Se concentrations as high as any of the Crow 

Creek samples (up to 14 ug/g). However, the lack of reference samples poses a greater problem for 

interpretation of the deformity endpoint. Whether due to toxic effects of Se (as asserted by FWS) or due 

to natural causes (or ‘unnatural’ causes such as fungal infection), some samples from Crow Creek had 

frequencies of ‘maximum fully normal’ larvae as low as 43%. For simplicity, I will express these rates as 

conservative (high) estimates of deformity rates – in this example up to 57%.

The Report used two groups of hatchery-spawned samples with low egg-Se levels (about 1 ug/g) 

as ‘method controls’, but these samples provide an inconclusive substitute for field reference samples. 

The hatchery fish presumably differed from wild fish in one or more respects, such as genotype, size/age, 

nutrition, and other environmental influences – factors that plausibly could have affected deformity rates. 

One of the two groups (SPC) is probably not appropriate for use either as controls or as reference 

samples, because they were not spawned and reared using the same methods used for the field-collected 

fish. These methodological differences may have contributed to the lower and less variable deformity 

rates in the SPC samples (14-18%), compared to the SC hatchery samples (9-51%). If the SC samples are 

considered to be suitable ‘surrogate reference’ samples (instead of the SPC samples used in the FWS 

analysis) there would be no discernible trend in the frequency of deformities between the SC hatchery 

samples (egg-Se ~1 ug/g) and the CC samples, which had egg-Se levels (6-14 ug/g) up to twice the EC10 

for deformities estimated by FWS.

The relatively high deformity rates for SPC hatchery samples raise concern about whether these 

samples accurately represent deformity rates in field-collected reference samples. Field reference 

samples from previous embryo-transfer Se toxicity studies with other salmonid species showed varying 

frequencies of deformities (as well as varying egg-Se concentrations): cutthroat trout, 0-25% deformed 

(0-10 ug/g [dwt]egg Se; Kennedy et al. 2000); cutthroat trout, mean 5.6% deformed (1.6 ug/g egg-Se; 

Hardy et al. 2010); brook trout, 0.7-2.1% deformed (mean egg-Se 1.24 ug/g; Holm et al. 2005), and 

rainbow trout, 5-42% deformed (mean egg-Se 5.8 ug/g; Holm et al. 2005. Of these species, only rainbow 

trout – the species with the highest ‘reference’ deformity rate – showed a significant relationship between 
35

egg-Se concentrations and deformity rates (EC10=23 ug/g). Conclusion: The lack of field-collected 

reference samples results in significant uncertainty about the normal background frequency of deformities 

in larval brown trout from the study area, but does not affect the survival endpoints.

Loss of swim-up larvae due to overflow

Large numbers of larvae were lost to overflow due to clogged drains during the post-swim-up 

stage of the study. Up to 50% of larvae were lost from 10 of 24 field replicates and from 2 of 12 hatchery 

(SPC) controls. These losses added substantial uncertainty to the calculation of two of the three primary 

endpoints of the study: overall survival (from hatch to 15 days post-swim-up) and rates of deformities. 

However, all the data from the Report (including samples with missing larvae) have considerable value 

because they are the only available Se toxicity data for brown trout, which is a dominant fish species in 

the watershed of the Smoky Hill mine and a member of a family that is relatively sensitive to selenium 

(Janz et al 2010). The FWS review simply discarded the data from the reps with lost larvae, which 

greatly simplified the data analysis. However, this strategy produced a sparse data set with no data in the 

range of egg-Se concentrations, 14 ug/g to 27 ug/g, that encompassed most published toxicity thresholds 

for other fish species. In contrast, the re-analysis by EPA used the available data from all the affected 

reps and calculated overall survival and frequency of deformities under different scenarios that illustrate 

the full range of uncertainty in these endpoints due to the missing larvae. 

The FWS approach has the advantage of only using only the most reliable data, but removal of 

replicates with missing larvae produced a much smaller dataset (14 instead of 24 field-collected samples) 

with a discontinuous distribution of egg-Se concentrations. The EPA analysis demonstrated that 

inclusion of the affected reps substantially affected concentration-response curves for both survival and 

deformities, even after making the most pessimistic assumptions about survival and deformity rates for 

lost larvae. Overall, the EPA analysis convincingly demonstrates the value of including data from reps 

with lost larvae. Most notably, two samples with missing larvae, LSV2C-008 and LSV2C-015, had EPAestimated maximum deformity rates of 33% and 41% at egg-Se concentrations of 17.7 and 20.5 ug/g, 

respectively, approximately three times greater than the EC10 for deformities estimated by FWS (6.5 

ug/g). EPA-modeled EC10s for survival and deformities did not differ greatly between optimistic and 

pessimistic scenarios, but EC10s for both endpoints differed substantially between EPA and FWS models. 

Conclusion: The loss of so many larvae indicates poor design of the apparatus and/or inadequate 

oversight by personnel. However, careful analysis of data from the replicates with missing larvae is 

acceptable – and is probably necessary to make reliable estimates of Se toxicity to brown trout to Se.

Effect of fungus infection on hatching, survival, and deformities

Although the Report and EPA describe the development and treatment of the fungal disease that 

infected the trout eggs during the incubation period, the possible impact of the fungus on the outcome of 

the test remains unclear. I am not familiar with this topic, but I found one reference that reported no 

difference in frequencies of deformities between rainbow trout eggs with moderate levels of fungal 

infection (15-27% infected) and eggs with low levels of infection due to prophylactic treatments (Arndt 

et al. 2001). One reported effect of the fungal infection was reduced egg hatching success, and hatching 

success was lower for the field-collected and SC hatchery eggs, which were held in the laboratory 

throughout the embryo-larval period and were most affected by fungus. However, variation in hatching 

success did not directly affect test endpoints, which were all determined from post-hatch samples in the 

EPEA analysis. The difference in deformity rates between the hatchery samples from SC (low fungus, 

low deformities) and SPC (high fungus, higher deformities) may reflect differences in fungal infection, 

but it may also reflect differences in methodologies used for the two groups of hatchery samples. The 

occurrence of fungus in both SPC and field samples, together with the overall consistency in rearing of 

these two groups, make the SPC samples more appropriate for use as method controls and as surrogates 

for field reference samples. Conclusion: There is no clear evidence that the fungal infection of eggs had 

significant effects on survival or rates of deformities in larvae.
36

Limitations of deformities data

I agree with both EPA and FWS that the Report’s presentation of the frequency of deformities 

separately for each class of deformity prevents the analysis of the data as combined frequency of 

deformities -- the percent of larvae with at least one type of deformity. The failure of the Report’s 

sponsor to provide such combined deformities data upon request of FWS, as well as Report’s use of the 

nonsensical ‘sum total normal’ endpoint, suggests that the deformities data were not recorded in a fashion 

that allowed calculation of the combined frequency of deformities. Most, but not all, previous studies of 

salmonid deformities (cited above) reported both combined frequencies and frequencies of each type of 

deformity. However, neither approach has been shown to be consistently more sensitive to selenium 

effects on salmonids. Only one previous study with salmonids was able to establish an threshold 

concentrations for larval deformities based on for egg-Se (in rainbow trout; Holm et al 2005), based on 

analyses of data for individual types of deformities. Conclusion: The available deformities data are 

limited, but are probably adequate to estimate Se toxicity thresholds for frequencies of larval deformities 

– either measured frequencies of individual deformities or estimated frequencies of combined deformities.

B. ALTERNATIVES FOR DATA ANALYSIS

Estimation of larval survival and deformities

EPA re-calculated data for all endpoints and explained how these calculations were intended to 

characterize the uncertainties caused by missing larvae and by the limitations of the deformities data. For 

survival, these calculations used an optimistic scenario (assuming missing fish had same survival as other 

larvae) and a pessimistic scenario (assuming all missing fish died). EPA estimated frequencies of 

deformities using six scenarios: three different formulas to estimate the frequency of deformities (ranging 

from ‘maximum fully normal’ to ‘minimum fully normal’), combined with optimistic and pessimistic 

scenarios for frequency of deformities in missing larvae (analogous to scenarios for survival data). These 

six scenarios were applied first to data for all deformities and then to data for ‘major’ deformities. 

Finally, the same six scenarios were applied to a composite endpoint that incorporated effects on both 

survival and frequency of deformities.

Overall -- across multiple endpoints -- the different scenarios had relatively little effect on EC10s. 

The pessimistic scenarios produced lower EC10s for full-test survival, but not for deformities, major 

deformities, or survival plus deformities. The pessimistic scenario for estimating combined deformity 

rates actually produced higher EC10s for both deformities and major deformities.

Comparisons among alternate data subsets 

The EPA analysis also examined the effect of including or excluding data from control groups 

and/or data from replicates with missing larvae on EC10s for all endpoints. The dataset analyzed in the 

Review, which consisted of field samples without missing larvae plus SC hatchery controls, was 

compared to all field samples (Field); to all field samples plus SPC hatchery controls (Field+SPC); and to 

all samples (All). The Review dataset produced the lowest EC10s for all endpoints analyzed. The 

deformities endpoint was much more sensitive for the Review data (range of EC10s, 6.5-14 ug/g, 

depending on the method used to calculate deformities) than for the other datasets (19-24 ug/g). For the 

other endpoints, EC10s for the Review dataset (12.9-18.5 ug/g) were closer to EC10s for the other 

datasets (16.4-23 ug/g).

Reliability of alternative EC10 estimates

The brown trout parental transfer Se toxicity study presented in the Report had several significant 

problems (discussed above) that raised doubts about the toxicity thresholds presented in the Report. The 

Review attempted to minimize the influence of these factors by excluding all ‘questionable’ samples. In 

contrast, the EPA analysis examined the entire dataset and attempted to quantify the uncertainties in larval 

survival and deformity rates. The alternative scenarios produced by EPA indicated that EC10s for 
37

survival and deformities were generally not sensitive to either the full range of possible outcomes for the 

missing larvae or to different decrees of association among the different types of deformity. However, 

EC10s were sensitive to the exclusion of the data from the field-collected samples with missing larvae. 

EC10s were consistently higher when these data were included in the analysis, even when worst-case 

scenarios for mortality or deformities of missing larvae were applied. These shifts in the concentrationresponse curves were primarily due to the influence of the two field-collected samples with missing 

larvae and egg-Se concentrations of 17.1 ug/g and 20.5 ug/g. Both of these samples had relatively high 

survival (worst-case estimate, 53-70%) and relatively low frequencies of deformities (worst-case 

estimate, 33-41%) at egg-Se concentrations 3 times greater than the deformities EC10 estimated from the 

Review dataset.
38

4. REFERENCES

Arndt, R.E., E.J. Wagner and M.D. Routledge. 2001. Reducing or withholding hydrogen peroxide 

treatment during a critical stage of rainbow trout development: Effects on eyed eggs, hatch, deformities, 

and fungal control. North Amer. J. Aquacult. 63: 161-166.

Besser, J.M., W.G. Brumbaugh, D.M. Papoulias, C.D. Ivey, J.L. Kunz, M. Annis and C.G. Ingersoll. 

2012. Bioaccumulation and toxicity of selenium during a life-cycle exposure with desert pupfish 

(Cyprinodon macularius). U.S. Geological Survey, Columbia Environmental Research Center, Columbia, 

MO, 40 pp. Accessed from http://pubs.usgs.gov/sir/2012/5033/ [Accessed May 2012].

deBruyn, A.M.H., B.G. Wernick, C. Stefura, B.G. McDonald, B.L. Rudolph, L. Patterson and P.M. 

Chapman. 2007. In situ experimental assessment of lake whitefish development following a freshwater oil 

spill. Environ. Sci. Technol. 41: 6983-6989.

Delos, C. 2014. Analysis of the brown trout selenium toxicity study presented by formation 

environmental and reviewed by U.S. Fish and Wildlife Service, 2014 Revision, addendum to: US EPA. 

2014. Re-analysis of the brown trout selenium toxicity study presented by formation environmental and 

reviewed by U.S. Fish and Wildlife Service, 2014 Revision. Office of Water, U.S. Environmental 

Protection Agency, Washington, DC.

Elliott, J.M. 1989. Wild brown trout Salmo trutta: an important national and international resource. Fresh.

Biol. 21(1): 1-5.

Erickson, R.J. 2010. Toxicity response analysis program, version 1.21. U.S. Environmental Protection 

Agency, National Health and Environmental Research Laboratory, Mid-Continent Ecological Division, 

Duluth, MN. Accessed from http://www.epa.gov/med/prods_pubs.htm [Accessed December 2010].

Formation Environmental. 2011. Appendix D: Brown trout laboratory reproduction studies conducted in 

support of development of a site-specific selenium criterion (227 pages). In: Technical Support 

Document: Proposed site-specific selenium criterion, Sage and Crow Creeks, Idaho. Prepared for the J.R. 

Simplot Company by Formation Environmental, Boulder, CO.

Hardy, R.W., L.L. Oram and G. Möller. 2010. Effects of dietary selenomethionine on cutthroat trout 

(Oncorhynchus clarki bouvieri) growth and reproductive performance over a life cycle. Arch. Environ.

Contam. Toxicol. 58: 237-245.

Holm, J. 2002. Sublethal effects of selenium in rainbow trout (Oncorhynchus mykiss) and brook trout 

(Salvelinus fontinalis), MA Thesis, Univ. Manitoba, Winnipeg, MB, Canada.

Holm, J., V. Palace, P. Siwik, G. Sterling, R. Evans, C. Baron, J. Werner and K. Wautier. 2005.

Developmental effects of bioaccumulated selenium in eggs and larvae of two salmonid species. Environ.

Toxicol. Chem. 24(9): 2373-2381.

Janz, D.M., D.K. DeForest, M.L. Brooks, P.M. Chapman, G. Gilron, D. Hoff, W.A. Hopkins, D.O. 

McIntyre, C.A. Mebane, V.P. Palace, J.P. Skorupa and M. Wayland. 2010. Selenium toxicity to aquatic 

organisms. (Pages 139-230). In: P.M. Chapman, W.J. Adams, M.L. Brooks, C.G. Delos, S.N. Luoma, 

W.A. Maher, H.M. Ohlendorf, T.S. Presser and D.P. Shaw (Eds.). Ecological Assessment of Selenium in 

the Aquatic Environment. SETAC Press, Pensacola, Florida, 

http://www.routledge.com/books/details/9781439826775/.
39

Krimsky S. 2005. The weight of scientific evidence in policy and law. Amer. J. Public Health. 95(Suppl 

1): S129-S136.

McDonald, B.G. and P.M. Chapman. 2009. The need for adequate QA/QC measures for Se larval 

deformity assessments: implications for tissue residue guidelines. Integ. Environ. Assess. Manag. 5(3): 

470–475. http://dx.doi.org/10.1897/IEAM_2009-005.1.

McDonald, B.G., A.M.H. deBruyn, J. Elphick, M. Davies, D. Bustard and P.M. Chapman. 2010. 

Development of a selenium tissue toxicity threshold for Dolly Varden char (Salvelinus malma). Environ.

Toxicol. Chem. 29: 2800-2805.

Kennedy, C., L. McDonald, R. Loveridge and M. Strosher. 2000. The effect of bioaccumulated selenium 

on mortalities and deformities in the eggs, larvae, and fry of a wild population of cutthroat trout 

(Oncorhynchus clarki lewisi). Arch. Environ. Contam. Toxicol. 39: 46-52.

Muscatello, J.R., P.M. Bennett, K.T. Himbeault, A.M. Belknap and D.M. Janz. 2006. Larval deformities 

associated with selenium accumulation in northern pike (Esox lucius) exposed to metal mining effluent. 

Environ. Sci. Technol. 40: 6506-6512.

Rudolph, B.L., C.K. Kennedy and I. Andreller. 2007. Report on the effects of selenium on westslope 

cutthroat trout reproduction and development in fish captured from Clode Pond and O'Rourke Lake. Elk 

Valley Selenium Task Force, Sparwood, BC, Canada.

Scholze, M., W. Boedeker, M. Faust, T. Backhaus, R. Altenburger and L.H. Grimme. 2001. A general 

best-fit method for concentration-response curves and the estimation of low-effect concentrations. 

Environ. Toxicol. Chem. 20(2): 223-458. http://dx.doi.org/10.1002/etc.5620200228.

Stephan, C.E., D.I. Mount, D.J. Hansen, J.H. Gentile, G.A. Chapman and W.A. Brungs. 1985. Guidelines 

for deriving numerical national water quality criteria for the protection of aquatic organisms and their 

uses. U.S. Environmental Protection Agency, EPA 822-R-85-100, NTIS PB85 227049, Duluth, 

Narragansett, and Corvallis. 98 pp. Accessed from http://epa.gov/waterscience/criteria/aqlife.html.

Taulbee, K., D. McIntyre and C.G. Delos. 2012. Analysis of the brown trout selenium toxicity study 

presented by Formation Environmental and reviewed by U.S. Fish and Wildlife Service. Great Lakes 

Environmental Center, Columbus, OH and Office of Water, U.S. Environmental Protection Agency, 

Washington, DC., 68 pp.

USEPA (United States Environmental Protection Agency). 1985. Guidelines for deriving numerical 

national water quality criteria for the protection of aquatic organisms and their uses. Washington, D.C. 

NTIS No. PB85-227049, 98 pages.

USEPA (United States Environmental Protection Agency). 2014. External peer review draft aquatic life 

ambient water quality criterion for selenium – freshwater. U.S. Environmental Protection Agency, Office 

of Water, EPA 822-P-14-001.

USFWS (United States Fish and Wildlife Services). 2012. Technical review: Smoky Canyon mine sitespecific selenium criterion report. U.S. Fish and Wildlife Service, Division of Environmental Quality, 

Environmental Contaminants Program, Alexandria, VA. Accessed from 

http://www.fws.gov/contaminants/pdf/ReviewSmokyCanyonMineSeleniumReport.pdf. [Accessed August

2012].
40

Wang, L. and R.J. White. 1994. Competition between wild brown trout and hatchery greenback cutthroat 

trout of largely wild parentage. North Amer. J. Fish. Manag. 14(3): 475-487. 

http://dx.doi.org/10.1577/1548-8675(1994)014<0475:cbwbta>2.3.co;2.

Warren-Hicks, W.J. and A. Hart (Eds.). 2010. Application of uncertainty analysis to ecological risks of 

pesticides. SETAC Press, Pensacola, FL.

Waters, T.F. 1999. Long-term trout production dynamics in Valley Creek, Minnesota. Trans. Amer. Fish.

Soc. 128(6): 1151–1162.

Zimmerman, J.K.H. and B. Vondracek. 2005. Interactions of slimy sculpin (Cottus cognatus) with native 

and nonnative trout: Consequences for growth. Can. J. Fish. Aquat. Sci. 63(7): 1526-1535.

Zimmerman, J.K.H. and B. Vondracek. 2007. Brown trout and food web interactions in a Minnesota 

stream. Fresh. Biol. 52(1): 123-136.