United

Enviro

Protec

Agenc

Su

In 

Pr

Av

Ju

d States 

onmental 

ction 

cy 

urvey 

Supp

referen

vailabi

ne 20

Supp

port of 

nce S

ility 

012

port D

Secti

Survey

ocum

on 31

 Notic

ent

6(b) S

e of D

tated

ata 
June 2012 Survey Support Document i 

Contents 

I.Introduction ...................................................................................................................... 1

II. Willingness to Pay Survey ............................................................................................. 5

A. Survey Format and Experimental Design ................................................................ 10

B. Sampling Frame ....................................................................................................... 18

C. Mail Survey Responses ............................................................................................ 21

D. Random Utility Model ............................................................................................. 22

E. Estimation of Willingness-to-Pay ............................................................................ 32

F. Method for Estimating Regional and National Benefits .......................................... 37

G. Northeast Non-Response Study, Weighted Model Results, and Sensitivity to Scope

 42
June 2012 Survey Support Document 1 

I. Introduction 

On April 20, 2011, EPA published proposed standards for cooling water intake 

structures at all existing power generating, manufacturing, and industrial facilities as part 

of EPA’s implementation of its responsibilities under section 316(b) of the Clean Water 

Act (CWA), at 76 FR 22174. EPA received voluminous comments and data submissions 

during the 90-day public comment period. After many commenters requested additional 

time to review the proposal, on July 20, 2011, EPA extended the comment period by an 

additional 30 days (76 FR 43230). 

As part of the proposal, EPA indicated it was in the process of developing a stated 

preference survey to estimate total willingness to pay (WTP) for improvements to fishery 

resources affected by impingement and entrainment (I&E) mortality from in-scope 

316(b) facilities. EPA acknowledged it did not have sufficient time to fully develop and 

implement this survey for the proposed regulation (76 FR 22174). EPA indicated its 

intent to issue a Notice of Data Availability (NODA) pending survey implementation and 

data analysis. This support document provides additional detail to the NODA that was 

signed on June 1, 2012, for publication in the Federal Register. 

In this NODA survey support document, EPA is providing additional preliminary 

data that may be relevant to the benefits of the rule, based on the results of a stated 

preference survey. Stated preference surveys are an attempt to determine the economic 

value of goods or services outside of the context of the marketplace. Simply described, a 

stated preference survey attempts to gauge the value of an item through questions 

designed to mimic consumer decision-making in actual markets. A stated preference 
June 2012 Survey Support Document 2 

survey, in this case, was used to measure values associated with ecosystem 

improvements. The stated preference survey estimates the value held by the public for 

ecosystem improvements based on the choices the surveyed members of the public make 

between hypothetical policy options and current conditions. In the proposal published in 

April 2011, EPA’s estimated benefits were partial estimates only. Specifically, the 

proposal included only a partial estimate of non-use values, or those values people may 

hold for an environmental improvement that are not associated with use of the resource 

(e.g., recreation). When there is no behavioral trail (Larson, 1993), that is, no observable 

behaviors (such as recreational trips) that can be analyzed to infer value, stated preference 

methods are the only way to measure values, especially non-use values (U.S. EPA 2010 

Guidelines for Preparing Economic Analyses). 

 Section II presents this new data and preliminary analysis for the 

Northeast, Southeast, Inland, and Pacific regions as well as the national survey. EPA 

presents a set of unweighted models that do not account for possible systematic variations 

between the populations of individuals that responded and did not respond to the surveys. 

The survey non-response assessment work has been completed for only the Northeast 

region; therefore EPA presents a weighted model which statistically adjusts for the 

differences among populations for that region. EPA has not yet completed the nonresponse assessments for the Pacific, Southeast, and Inland regions and the national 

survey. The remainder of this section provides an overview of this Survey Support 

Document. The first step in developing a stated preference survey is to specify the 

question format. Question design is usually developed using focus groups to test wording 

and attribute selection and ensure that respondents understand and are not cognitively 
June 2012 Survey Support Document 3 

burdened by the question format. Attributes should also link directly to the policy to be 

analyzed. Choice experiments are designed to mimic consumer decision–making in 

actual markets where goods comprise a bundle of attributes and consumers make 

tradeoffs among those attributes when selecting bundles for purchase. Once the question 

format is developed, the experimental design, which is the plan for varying attribute 

levels across questions within a survey and across survey versions, must be specified. 

Following recommendations in the stated preference literature, these attribute levels vary 

according to realistic ecological and policy expectations (see Section II.A for more 

details). 

The next steps are to determine the mode of survey delivery (e.g., mail, phone, or 

web-based), and develop the associated sample frame. The sample frame is the 

population from which potential respondents are selected, in this case, at random (see 

Section II.B for more details). EPA chose to conduct a mail survey, and used a list of all 

postal addresses as the sample frame. The address sample was drawn from database 

which covers 97% of residences in the United States. The mail survey approach avoids 

potential sampling biases associated with incomplete coverage of landline and cellphone 

databases. Upon return of surveys, EPA entered responses into a database (see Section 

II.C for more details). EPA has completed fielding the main versions of the four regional 

and the national surveys, by virtue of achieving the target response rate of 30%. 

EPA has also estimated preliminary regional and national models based on data 

from respondents. In general, EPA expected to observe variations in surveyed 

individuals’ responses across the regions, due in part to differences in tastes and 

preferences of respondents, and to aspects of the experimental design (e.g., baseline 
June 2012 Survey Support Document 4 

levels of the attributes vary across the regions). Although EPA has not yet conducted 

scope and validity testing, certain expected patterns have been observed (e.g., there is 

variation across the regions in the importance respondents place on the different 

attributes). 

Once regional and national models have been appropriately estimated and subject 

to validity testing and peer review, EPA may then use the reduced fish mortality from 

cooling water intake structures to predict average household willingness-to-pay 

associated with each of the regions. At the time this supporting document was prepared, 

EPA had produced preliminary estimates of average willingness-to-pay per household per 

percentage point improvement in each of the attributes, based on unweighted models (see 

Section II.E for more details). For example, the average dollar value per household per 

percentage point of reduced fish mortality at cooling water intake structures ranges from 

$0.75 and $0.78 in the Southeast and Inland regions, respectively, to $1.12 and $1.13 in 

the Northeast and national versions, respectively, to $2.52 in the Pacific region. These 

preliminary results suggest that respondents are responding rationally, given that fish 

mortality in the Inland region is generally lower than the coastal regions, and that the 

national average is within the range of the regional results, based on responses from 

different respondents. 

EPA is also conducting a non-response study for each version of the survey, to 

account for the possibility that respondents are fundamentally different from nonrespondents. EPA would use the non-response study results to develop weights that 

correct for any differences, reducing the weight placed on overrepresented respondent 

groups, while increasing the weight placed on any underrepresented respondent groups. 
June 2012 Survey Support Document 5 

See Section II.G for details on the non-response study for the Northeast region of the 

survey (the only region for which the non-response study has been completed to date). 

EPA currently is still fielding the non-response studies for the other three regional and 

national versions of the survey. EPA intends to complete weighted models for the 

remaining regions and the national surveys. After completing the non-response studies 

for all regions, reviewing public comment, and conducting additional scope and validity 

testing, EPA will present a more complete set of stated preference survey materials for an 

external peer review. EPA will then determine what role, if any, the monetized benefits 

based on the stated preference survey should play in EPA’s assessment of benefits for the 

final rule, and will include the complete results in its assessment as appropriate. 

II. Willingness to Pay Survey 

In this supporting material, EPA is documenting the availability of data collected 

from a stated preference survey designed to facilitate the estimation of households’ 

willingness to pay to reduce the number of fish impinged or entrained in cooling water 

intake structures. Stated preference methods provide a non-market approach to 

quantifying values associated with ecosystem improvements, such as increased protection 

of aquatic species or the restoration of habitats with specific attributes. These methods 

rely on an analysis of responses to survey questions through which individuals state 

information about their values. In contrast, EPA’s estimated benefits at proposal were 

partial estimates only. Specifically, the proposal included only a partial estimate of nonuse values, that is, values individuals may hold for an environmental improvement that 

are not tied to use of the resource (e.g., recreation). 
June 2012 Survey Support Document 6 

Estimation of monetized non-use benefits is challenging, since market proxies are 

generally not available, and in the absence of such proxies, they can only be estimated by 

using either stated preference methods or benefits transfer based on prior stated 

preference results. For this reason, non-use benefits are often discussed qualitatively 

instead of attaching monetized values to them. Today’s notice presents data collected 

from a stated preference study (EPA ICR # 2402.01) that EPA conducted regarding total 

(use plus non-use) benefits from reductions in fish mortality at cooling water intake 

structures. EPA’s peer-reviewed guidelines for benefits analysis (U.S. EPA 2010, pp. 7-

41, DCN 11-4712) recognize “advantages of [stated preference] methods includ[ing] 

their ability to estimate non-use values and to incorporate hypothetical scenarios that 

closely correspond to a policy case.” 

The results presented in this document are estimated based on a choice 

experiment. Choice experiments, also called choice models, are a stated preference 

technique in which individuals’ values are estimated based on their choices over a set of 

hypothetical but realistic policy options. 

Stated preference methods have “... been tested and validated through years of 

research and are widely accepted by ... government agencies and the U.S. courts as 

reliable techniques for estimating non-market values” (Bergstrom and Ready 2009, p. 

26). EPA’s own peer-reviewed Guidelines for Preparing Economic Analysis (US EPA 

2010, DCN 11-4712) indicate that the use of stated preference study data, when the study 

is conducted properly in accord with best current practices, is the only potential method 

for monetizing non-use values. Likewise, OMB’s Circular A-4 notes that stated 

preference results “have also been widely used in regulatory analyses by Federal 
June 2012 Survey Support Document 7 

agencies” (p. 22), and that “there is no mechanical formula that can be used to determine 

whether a particular study is of sufficient quality to justify use in regulatory analysis” (p. 

23). OMB also cautions that “a number based on a poor quality study is not necessarily 

superior to no number at all” (p. 24). 

EPA recognizes that several issues have been raised regarding the estimation of 

welfare values from stated preference surveys. For example, the stated preference study 

discussed in this document creates a hypothetical market where respondents are asked to 

reveal their values for increases in “fish saved” in conjunction with increases in fish 

populations (total and commercial) and aquatic conditions by voting for or against 

alternative hypothetical policies that would regulate cooling water intake structures and 

that would impose increases in annual household cost of living. The issue of whether 

respondents are capable of respecting hypothetical budget constraints knowing that their 

responses to the survey would not compel them to incur any costs is a concern that has 

been cited as a reason to question t the results of stated preference studies. The 

hypothetical nature of the market has raised questions as to whether this type of 

elicitation accurately reveals and elicits WTP associated with the good being considered. 

Substantial research has been conducted over the past two decades on 

hypothetical bias in stated preference surveys. While many studies have found evidence 

of hypothetical bias (List and Gallet 2001), a recent meta-analysis indicates that 

“hypothetical bias in SP studies may not be as important” as some have argued 

previously (Murphy et al. 2005). This mirrors similar findings in prior studies that 

compare hypothetical and actual referenda (see discussion in Johnston 2006). EPA used 

all available means for mitigating hypothetical bias in designing and conducting the 
June 2012 Survey Support Document 8 

survey. Stated preference surveys also require the provision of information to enable 

respondents to comprehend the potential implications of their hypothetical choices. For 

example, in this case, respondents may not be aware that cooling water intake structures 

can potentially kill large numbers of fish, eggs, and larvae, or that the vast majority of 

those organisms are species that provide no consumptive use (e.g., commercial or 

recreational fishing) to humans. Even if they are aware of this issue in a general way, it 

is unlikely that most respondents will have previously considered what preserving those 

species is worth to them. In order to elicit informed responses, it is necessary to provide 

information to respondents about the general context and scope of the issue. Following 

standard practice, EPA pretested the information provided to respondents in focus groups 

and cognitive interviews to determine what quantity and types of information were 

required by respondents in order to feel confident and well-informed in their responses 

(DCN 11-4710). For example, in the introductory materials accompanying the four 

regional and national stated preference surveys, EPA presents the number of “young 

adult fish” (also called “age-one-equivalents”) that are lost in coastal and fresh waters 

due to cooling water use, and that these losses include eggs and larvae. Without this 

educational material, respondents to the survey might not otherwise realize that reported 

effects on “fish saved per year” in the valuation questions partially result from reduced 

mortality of eggs and larvae. The presentation of this type of background information, if 

not properly vetted in the survey instrument development process, could result in 

focusing respondent attention on particular environmental amenities to the exclusion of 

other market and non-market goods that may also be important to some respondents’ 

decision making with regard to the choice questions. 
June 2012 Survey Support Document 9 

Consistent with established best practices for stated preference surveys, EPA has 

sought to minimize possible biases by careful and thorough construction and testing of 

the survey instrument. The Agency recognizes that potential biases may still remain and 

may influence the results of the study. While in EPA’s view, the study incorporates 

current best professional practice in the conduct of stated preference studies, EPA 

acknowledges that the results of any empirical study depend on the methodology applied. 

EPA has not yet completed its statistical analyses of these survey data and therefore has 

not determined the role that the monetized benefits assessed through the stated preference 

survey will play in the analysis of benefits for the final rule. 

At the time this supporting material was prepared, EPA had finished fielding all 

five versions of the main mail survey (four regional and one national). EPA undertook 

the Northeast version in advance of the other versions as a pilot study to inform potential 

changes to other survey versions, as described in the ICR for the 316(b) stated preference 

survey (EPA ICR # 2402.01) and as recommended in published guidance for stated 

preference survey design (Arrow et al. 1993; Bateman et al. 2002). EPA notes that the 

stated preference survey regions differ from the benefits regions used in the 

Environmental and Economic Benefits Assessment (EEBA) document for the proposed 

rule.1

 This section describes the methods used by EPA to develop and implement the 

316(b) stated preference survey and presents preliminary benefits estimates for each 

survey version (Northeast, Southeast, Pacific, Inland, and National). EPA uses a 

statistical model estimated based on the survey results to estimate benefits to households 

for ecological improvements. EPA emphasizes that benefits estimates presented here are 

 

1

 See Exhibit II-3 for a list of the states included in each survey region. 
June 2012 Survey Support Document 10 

preliminary. Results are presented separately by survey version. If EPA includes the 

quantified benefits derived from the stated preference survey in the final benefits 

analysis, the Agency will perform additional analysis comparing the results of the 

regional survey versions to the results of the national survey version. This additional 

analysis will allow EPA to look at the impacts of program size (regional vs. national) on 

willingness to pay and to consider the implications of any differences for the validity of 

results. For the final benefits analysis, EPA may present a range for the total national 

benefit estimates produced by the stated preference research for the final rule. 

Alternatively, EPA may decide to consider non-use benefits qualitatively and/or by using 

benefits transfer, as was done for the proposed rule. 

A. Survey Format and Experimental Design 

Stated preference surveys, in general, ask questions that elicit individuals’ stated 

values for carefully specified changes in an environmental amenity (Freeman 2003). This 

value is typically estimated in terms of willingness to pay (WTP), defined as the 

maximum amount of money (or some other commodity) that an individual or household 

would be willing to give up in exchange for a specified environmental change, rather than 

go without that change. Various question formats have been used in the stated preference 

literature to elicit stated WTP. Some types of stated preference surveys ask respondents 

to express their WTP using open-ended questions, payment cards, or bidding games. 

Increasingly, however, these original types of stated preference surveys have been 

replaced in the literature by methods grounded in random utility models (Hanemann 

1984), in which respondents express their WTP through choices over alternative policy 
June 2012 Survey Support Document 11 

options. Advantages of these choice-based methods include similarity to familiar 

referenda or market choice contexts, in which individuals choose among alternative 

bundles of attributes or commodities at different costs (Freeman 2003). These methods 

are intended to reduce the hypothetical choice bias that might result from asking survey 

questions versus assessing willingness to pay through market transactions or referenda. 

For the 316(b) stated preference survey, EPA followed well-established choice 

experiment methodology (Adamowicz et al. 1998; Louviere et al. 2000; Bennett and 

Blamey 2001; Bateman et al. 2002). Under the choice experiment (or choice modeling) 

format, respondents are presented with a set of multi-attribute alternatives and asked to 

select their preferred alternative, much as one might choose a preferred option in a public 

referendum. Choice experiments have been applied to assess WTP for ecological 

resource improvements of a type similar to those at issue in the 316(b) policy case (e.g., 

Bennett and Blamey 2001; Hanley et al. 2006a, b; Hoehn et al. 2004; Johnston et al. 

2002, 2011a, b; Milon and Scrogin 2006; Morrison and Bennett 2004; Morrison et al. 

2002; and Opaluch et al. 1999). Choice experiments allow survey respondents to express 

WTP for a wide range of different potential outcomes, differentiated by their attributes. 

This enables EPA to isolate the marginal effects of different potential policy outcomes on 

stated choices and hence, on estimated WTP. EPA can thereby estimate benefits for a 

wider range of potential policy outcomes. This is a primary factor distinguishing choice 

experiments from older forms of stated preference analysis, in which stated WTP is 

typically contingent upon a single specification of ecological effects. 

Following well-established methods, respondents in the choice experiment are 

presented with two alternative hypothetical policy options and asked to choose (or vote 
June 2012 Survey Support Document 12 

for) the policy that they would prefer. Respondents may also choose to reject both 

policies and retain the status quo. The underpinning theoretical model is adapted from a 

standard random utility specification in which household h chooses among three choice 

options (j=A,B,N), including two multi-attribute policy options (A, B) and a fixed “no 

policy” status quo (N) that includes no policy changes and zero cost to the household. 

Each choice option reflects a hypothetical but feasible outcome under various 316(b) 

regulatory alternatives. The effects of the policy options are described in terms of a 

household cost and four environmental endpoints, or attributes: (a) commercial fish 

populations, (b) fish populations (all fish), (c) fish saved per year, and (d) condition of 

aquatic ecosystems. The definition of each attribute is presented in Exhibit II-5. Values 

are reflected in the survey by individuals’ willingness to “vote” for policies that would 

increase their cost of living, in exchange for specified changes in the four environmental 

attributes. Other questions in the survey elicit information including whether the 

respondent is a user of the affected aquatic resources, household income, and other 

respondent demographics.2

Following established practices, EPA used an experimental design to generate 

multiple unique combinations of policy options for different respondents to compare. 

Respondents were presented with three separate policy questions in the survey, each with 

a specific combination of policy options. The experimental design specifies how attribute 

levels were “mixed and matched” within choice questions, thereby developing an 

 

2

 The four environmental attributes were designed based on the Johnston et al. (2011a,b; 2012) 

Bioindicator-Based Stated Preference Valuation (BSPV) method which was developed to promote 

ecological clarity and closer integration of ecological and economic information within SP studies. This 

methodology was developed in part to address the EPA Science Advisory Board’s call, in its May 2009 

report, Valuing the Protection of Ecological Systems and Services: A Report of the EPA Science Advisory 

Board, for improved quantitative linkages between ecological services and economic valuation of those 

services. 
June 2012 Survey Support Document 13 

empirical data framework with appropriate statistical properties to allow for analysis of 

respondent’s choices (Louviere et al. 2000). EPA applied a fractional factorial 

experimental design representing a subset of all possible combinations of environmental 

attributes and household cost. This allows efficient estimation of particular effects of 

interest (Louviere et al. 2000) and reduces the cognitive burden faced by respondents 

(Holmes and Adamowicz 2003). 

The fractional factorial experimental design was generated using a D-efficiency 

criterion for main effects estimation (Kuhfeld 2010; Kuhfeld and Tobias 2005). This 

more efficient design enables model coefficients, and hence, estimated willingness–topay, to be estimated with greater precision; i.e., lower standard errors or variability, for 

any given number of observations. It also minimizes correlation between attributes across 

survey questions (i.e., attributes do not “move together” across different survey 

questions), so that the unique effect of each attribute on respondents’ choices, and 

ultimately, values, can be isolated.3

 

Exhibit II-1 presents the set of attribute levels that are used across the option 

pairs. The resulting experimental design is characterized by 72 unique Option A vs. 

Option B pairs, each corresponding to a choice question defined by an orthogonal 

(independent) array of attribute levels for the two policy options. Following guidance 

from the literature, EPA designed the attribute levels to illustrate realistic policy 

scenarios that “span the range over which we expect respondents to have preferences, 

and/or are practically achievable” (Bateman et al. 2002, p. 259). Choice scenarios 

 

3

 EPA removed dominated pairs where one option is superior to the other in all attributes. Focus groups 

showed that respondents react negatively and often protest when offered dominated pairs. Given that such 

choices provide negligible statistical information compared to choices involving non- dominated pairs, they 

are typically avoided in choice experiment statistical designs. 
June 2012 Survey Support Document 14 

represent each ecological attribute in relative terms with regard to upper and lower 

reference conditions; i.e., best and worst possible in the attribute, as defined in survey 

informational materials. The surveys also present the cardinal basis for relative scores 

where applicable (e.g., respondents were asked to evaluate changes in fish saved per year 

as a percentage of current estimated mortality, but those changes were also illustrated in 

terms of numbers of age-one equivalent fish).4 Relative scores represent percent progress 

towards the upper reference condition (100%), starting from the lower reference 

condition (0%). In interpreting the results, it is useful to keep in mind that while three of 

the attributes spanned a relatively narrow range of percentage values reflecting realistic 

ecological expectations (e.g., commercial fish populations differing by no more than six 

percentage points from the baseline), the “fish saved per year” attribute, which was 

ultimately used to estimate household WTP for the policy options, was presented in a 

way that spanned a much larger range (e.g., up to 95 percentage points). Again, this 

reflects the expected range of potential reductions based on available technology 

performance. Allowing the range of variables to vary according to realistic ecological and 

technological expectations, is recommended practice in stated preference design 

(Bateman et al. 2002). 

 

4

 Age-one equivalents, in addition to providing a way to standardize losses so that losses could be 

compared among species, years, facilities, and regions, is a convenient way to express losses of all life 

stages, including fish eggs and larvae, as numbers of individual fish. 
June 2012 Survey Support Document 15 

Exhibit II-1—Attribute Levels Assigned Across Policy Options and Survey Versions 

Attribute Baseline 

(Status Quo)a

Max Change 

Assigned Attribute Levels Assigned to Option A vs. Option B Pairs 

1 2 3 4 5 6 

Commercial Fish Populations (Score showing the overall health of commercial and recreational fish populations) 

Northeast 42% 6% 43% 45% 48% - - -

Southeast 39% 6% 40% 42% 45% - - -

Pacific 56% 6% 57% 59% 62% - - -

Inland 39% 6% 40% 42% 45% - - -

National 51% 6% 52% 54% 57% - - -

Fish Populations (all fish) (Score showing the estimated size of all fish populations compared to natural levels without 

human influence) 

Northeast 26% 4% 27% 28% 30% - - -

Southeast 24% 4% 25% 26% 28% - - -

Pacific 32% 4% 33% 34% 36% - - -

Inland 33% 4% 34% 35% 37% - - -

National 30% 4% 31% 32% 34% - - -

Fish Saved per Year (Score showing the reduction in young fish lost compared to current levels) 

Northeast 0% 95% 5% 50% 95% - - -

Southeast 0% 90% 25% 55% 90% - - -

Pacific 0% 95% 2% 50% 95% - - -

Inland 0% 95% 55% 75% 95% - - -

National 0% 95% 25% 55% 95% - - -

Aquatic Ecosystem Condition (Score showing the ecological condition of affected areas, compared to the most natural 

waters in the region) 

Northeast 50% 4% 51% 52% 54% - - -

Southeast 68% 4% 69% 70% 72% - - -

Pacific 51% 4% 52% 53% 55% - - -

Inland 42% 4% 43% 44% 46% - - -

National 53% 4% 54% 55% 57% - - -

Household Costs (The increase in annual household cost, in unavoidable price increases) 

Northeast $0 $72 $12 $24 $36 $48 $60 $72

Southeast $0 $72 $12 $24 $36 $48 $60 $72

Pacific $0 $72 $12 $24 $36 $48 $60 $72

Inland $0 $72 $12 $24 $36 $48 $60 $72

National $0 $72 $12 $24 $36 $48 $60 $72 a Each question includes a “no policy” option, characterized by the baseline levels for each attribute and a household cost 

of $0. 

The estimates of “fish saved” in the valuation questions are based on EPA’s 

estimate of age-one-equivalent (A1E) losses due to cooling water intake structures at 

baseline; a metric used by EPA to convert all I&E mortality losses into equivalent 

numbers of one-year-old fish. This conversion allows losses to be compared among 

species, years, facilities, and regions. Exhibit II-2 shows the composition of estimated 
June 2012 Survey Support Document 16 

national A1E losses at cooling water intakes at baseline in terms of the various life stages. 

The introductory materials describe the age classes impacted due to cooling water intakes 

and the “fish saved” metric as “young fish lost compared to current levels.” The A1E 

terminology is not used specifically within the valuation questions themselves, although 

pre-testing during focus groups and cognitive interviews suggested that participants 

understood the “fish saved” attribute and the concept of “young fish” as reflecting initial 

losses of eggs and other juvenile life stages. Page three of the survey booklet includes 

introductory materials that specify the proportion of “fish saved” that are and are not 

commercial or recreational species. 

Exhibit II-2—Composition of Baseline National CWIS Fish Losses a

Fish Group 

Organisms Lost A1E Losses 

Count 

(in millions) % Count (in millions) % 

By Life Stage 

Egg 387,199.2 73.2% 28.6 1.3% 

Larvae 138,740.3 26.2% 557.5 25.5% 

Juvenileb 2,225.7 0.4% 491.6 22.5% 

Adult 731.8 0.1% 1,110.9 50.8% 

Total 528,897.0 100.0% 2,188.6 100.0% 

By Use Category 

Forage 360,651.0 68.2% 1,654.5 75.6% 

Commercial & 

Recreational 168,245.9 31.8% 534.1 24.4% 

Total 528,897.0 100.0% 2,188.6 100.0% 

a

 Totals are based on the sum of losses within all survey regions. 

b

 The juvenile stage begins after fish larva undergo metamorphosis following absorption of the yolk sac. 

The metamorphosis is complete when all fin rays are present and the process of squamation (growth of fish 

scales) has begun. In our model, juvenile stages end at age 1. 

It is standard practice to include more than one choice question in each survey, 

thus increasing the information obtained from each respondent (Poe et al. 1997; Layton 

2000). EPA randomly assigned the 72 option pairs to 24 distinct versions for each of the 
June 2012 Survey Support Document 17 

four regional surveys and the national survey, with three option pairs (i.e., choice 

questions) per survey booklet. See the ICR supporting statement (EPA ICR # 2402.01) 

for additional detail on the experimental design. 

Following recommended methods for stated preference survey design (cf. Arrow 

et al. 1993; Bateman et al. 2002; Bennett and Blamey 2001; Kaplowicz et al. 2004), the 

survey instrument was pre-tested extensively in six focus groups, with eight to ten 

participants each, and a set of eight one-on-one cognitive interviews (EPA ICR # 2090-

0028). Each cognitive interview included only one participant. This allowed in-depth 

exploration of the cognitive processes used by respondents to answer survey questions, 

without the potential for interpersonal dynamics to sway respondents’ comments 

(Kaplowicz et al. 2004). Focus groups and cognitive interviews also included questions 

following the verbal protocols suggested by Schkade and Payne (1994), in which 

respondents were asked to talk through the process they used to answer choice questions. 

Within the focus group and cognitive interview format, the moderator first asked the 

participants to complete a draft survey questionnaire. The moderator then led a general 

conversation which took the group/individual through a series of debriefing questions. 

During debriefing, the moderator asked focus group and cognitive interview participants 

about their reactions to the survey format and content, whether the survey questions were 

clear, whether the background information presented in the survey or introductory 

materials was sufficient and how participants interpreted it, what went through 

participants’ minds when they read survey questions, and what their motivations were for 

responding the way they did to the survey. 
June 2012 Survey Support Document 18 

The participants comments and feedback provided important information on such 

concerns as (1) whether questions and survey information were readily understood, (2) 

whether respondents were interpreting questions similarly to how EPA interprets them, 

(3) whether responses or survey interpretations showed any evidence of heuristics or 

survey biases including hypothetical bias, (4) whether respondents were addressing 

choice questions in a manner commensurate with utility maximization and neoclassical 

WTP estimation, and (5) whether respondents were following instructions provided in the 

survey instrument and responding to questions accordingly. Focus group participants’ 

responses to the survey choice questions could not be included in model estimation 

because the draft surveys completed during pre-testing differ somewhat from the final 

survey. EPA modified the survey several times based on the results of these pre-tests, to 

help minimize potential biases, and to help ensure shared and accurate interpretation of 

survey language by the respondents. The amount of pre-testing conducted for stated 

preference surveys varies within the literature and tends to be related to the complexity of 

the survey instrument. EPA believes that the amount of time and number of focus groups 

the Agency applied in survey design compares quite favorably to stated preference 

analyses in the peer reviewed literature. 

B. Sampling Frame 

EPA designed the 316(b) stated preference study as a household mail survey. The 

mail survey approach allowed EPA to draw the survey sample from a nearly 

comprehensive database of residences in the U.S. and avoid potential sampling biases 

associated with the incomplete coverage of landline and cellphone databases. EPA 
June 2012 Survey Support Document 19 

stratified households based on the geographic boundaries of four regions: Northeast, 

Southeast, Inland, and Pacific. As noted previously, these regions differ from the 316(b) 

benefits regions used in the EEBA for the proposed rule. EPA developed target sample 

sizes for each region to provide statistically robust results while minimizing the cost and 

burden of the survey to individual respondents.5

 The target sample sizes refer to 

completed mail surveys. A larger number of households must be mailed surveys because, 

as is the case for all mail surveys, only a portion of households that receive a survey will 

return a completed mail survey. 

 EPA selected a total target sample of 2,000 completed surveys across all four 

regional surveys to provide estimates of population percentages with a margin of error 

ranging from 3.6 to 5.8 percentage points at the 95% confidence level. These 2,000 

surveys were allocated across the four regions based on the number of households in each 

region relative to the total number of household in the continental U.S. In addition, a 

minimum number of completed surveys were required for each region. Monte Carlo 

experiments indicate that approximately 6 to 12 completed responses are required for 

each profile (unique set of choice options) in order to achieve large sample statistical 

properties for choice experiments (Louviere et al. 2000, p. 104, citing Bunch and Batsell 

1989). As described previously, the experimental design includes 72 option profiles. 

Following this guidance, the experiment design will require 12 completed surveys for 

each of the 72 profiles, for a total of 864 profile responses per region (72×12=864). A 

 

5

 EPA included three choice questions within each survey, to increase information obtained from each 

respondent. It is standard practice within choice experiment and dichotomous choice contingent valuation 

surveys to include more than one choice question in each survey (Poe et al. 1997; Layton 2000). Including 

more than three choice questions may have negatively affected the response rate by increasing burden on 

respondents and including fewer would have increased survey costs by requiring additional households. 
June 2012 Survey Support Document 20 

minimum of 288 completed surveys are required for each region because each survey 

version includes 3 profiles (864÷3=288). 

The allocation of the 2,000 completed surveys across the four regions resulted in 

target sample sizes of 417 for the Northeast version, 562 for the Southeast version, 289 

for the Pacific version, and 732 for the Inland version. EPA also conducted a national 

version of the mail survey with a target sample size of 288 completed surveys. EPA 

mailed the survey to 7,840 households in total, anticipating a response rate of 30 percent. 

Exhibit II-3 presents the states included in each region, the total number of households in 

each region, the target number of completed surveys, and the number of surveyed 

households for each survey version. 

Exhibit II-3—Target Sample Sizes and Number of Mailed Surveys by Survey Region 

Survey Region State Included Number of 

Households 

Target Sample 

Size a,b

Number of 

Surveyed 

Householdsc

Northeast CT, DC, DE, MA, MD, ME, 

NH, NJ, NY, PA, RI, VT 

23,281,296 417 1,440

Southeast AL, FL, GA, LA, MS, NC, 

SC, TX, VA 

31,378,122 562 1,920

Pacific CA, OR, WA 40,852,983 289 1,040

Inland AR, AZ, CO, ID, IA, IL, IN, 

KS, KY, MI, MN, MO, MT, 

ND, NE, NM,NV, 

OH,OK,SD,TN, UT, WI, 

WV, WY 

16,158,206 732 2,480

Total for Regional 

Survey Versions 

U.S. (excluding AK and HI) 111,670,607 2,000 6,880

National Survey 

Version 

U.S. (excluding AK and HI) 111,670,607 288 960

a

 Target sample sizes presented here refer to completed mail surveys. b

 The sample is allocated to each region in proportion to the total number of households in that region, with at least 288 

completed surveys in each region, the number required to estimate the main effects and interactions under an 

experimental design model. 

c

 The number of intended completed questionnaires for each survey region was rounded up so that the same number 

of households received each of the 24 survey versions. 

EPA used multiple preview and reminder mailings to promote a high response 

rate and minimize the potential for non-response bias. This approach follows Dillman et 

al. (2009), which is among the most definitive sources for survey logistics management. 
June 2012 Survey Support Document 21 

Households were selected from the U.S. Postal Service Digital Sequence File (DSF) of 

residences which, in total, covers 97% of residences in the United States. EPA is also 

conducting a follow-up study of households that did not return a completed mail survey. 

The purpose of the non-response follow-up study is to identify whether survey nonrespondents are fundamentally different than survey respondents by asking them a few 

demographic and attitudinal questions. Refer to Section II.G below and the ICR (EPA 

ICR # 2402.01) for additional details regarding the non-response study and sampling 

design. 

C. Mail Survey Responses 

At the time this supporting material was prepared, EPA had received a total of 

2,313 completed mail surveys across all versions. Exhibit II-4 summarizes the number of 

completed surveys received and the response rate (minus undeliverable surveys across 

the survey versions. The average response rate across all versions was 33 percent. This 

response rate is comparable to various other recent mail surveys in the stated preference 

literature (e.g., Hanley et al. 2006; Johnston and Duke 2009; Johnston and Bergstrom 

2011; Boyle and Ozdemir, 2009). 

Exhibit II-4—Completed Surveys Received and Response Rates by Survey Version 

Survey Version Households 

Surveyed 

Completed Surveys 

Received Response Ratea

Northeast 1,440 421 31%

Southeast 1,920 506 30%

Pacific 1,040 311 32%

Inland 2,480 787 36%

National Survey Version 960 288 34%

a

 The number of undeliverable surveys was subtracted from surveys mailed when calculating the 

response rate for each survey region. Undeliverable surveys are those surveys that were returned to 

sender. 
June 2012 Survey Support Document 22 

Analysis of the survey data across all four regions and the national survey 

indicates that respondents appear to have been evaluating trade-offs between costs and 

benefits of policy options presented to them, and that WTP is responsive to scope (i.e., 

the quantity of environmental improvements across different attributes). Responses also 

reveal, as suggested in the focus groups, that respondents appeared to understand and 

distinguish between different types of outcomes from 316(b) regulation. About 90 

percent of respondents answered the choice experiment questions (questions 4, 5, and 6 

of the survey). Question 8 of the survey asked respondents to rate their understanding of 

the survey material, with only 14 percent disagreeing when asked whether the survey 

material was easy to understand. They were also confident in their responses to the 

survey questions. Seventy-one percent of respondents strongly agreed or agreed when 

asked if they were confident in their responses to the survey questions. The vast majority 

indicated that they would answer the same way if parallel questions were asked in a 

binding referendum with less than two percent of respondents indicating otherwise. 

About 75 percent of mail survey respondents were under age 65 and the majority 

of those completing the survey (63 percent) were male. About 87 percent of respondents 

selected “white” for racial category. For additional information on the demographic 

characteristics of respondents see EPA’s memorandum to the 316(b) rulemaking record 

(DCN-4524). 

D. Random Utility Model 

EPA’s analysis of the 316(b) stated preference survey data is grounded in the 

random utility model of Hanemann (1984) and McConnell (1990). The use of the random 
June 2012 Survey Support Document 23 

utility model is standard in the stated preference literature for attribute-based experiments 

such as choice experiments (Bennett and Blamey 2001; Bateman et al. 2002). Under the 

random utility model, “utility is the sum of systematic [or observed] and random [or 

unobserved] components” (Holmes and Adamowicz 2003, p. 189). The individual 

choices are systematic (i.e., deterministic) while the random component reflects 

researcher uncertainty about preferences, among other things (Holmes and Adamowicz 

2003). It is applied extensively within stated preference research, and allows well-defined 

welfare measures (i.e., WTP) to be derived from choice experiment models (Bennett and 

Blamey 2001; Louviere et al. 2000). 

The random utility models for all four regions and the national survey were 

estimated using maximum likelihood mixed logit. The mixed logit model is an approach 

for modeling preference heterogeneity based on the assumption that people’s preferences 

are randomly distributed and that heterogeneity in population preferences can be captured 

by estimating the mean and variance of the random parameter distribution (Holmes and 

Adamowicz 2003). As described by Henscher and Greene (2003), “the mixed logit 

model offers an extended framework within which to capture a greater amount of 

behavioral choice making. Broadly speaking, the mixed logit model aligns itself much 

more closely with reality than most discrete choice models. This is because every 

individual has their own inter-related systematic and random components for each 

alternative in their perceptual choice set(s)” (p. 170). It is a highly flexible model that 

“obviates the three limitations of standard logit by allowing for random taste variation, 

unrestricted substitution patterns, and correlation in unobserved factors over time” (Train 

2009, p.134). 
June 2012 Survey Support Document 24 

The mixed logit model allows for the possibility of preference heterogeneity but 

cannot attach specific parameter values to particular individuals. That is, the mixed logit 

(ML) model relaxes the assumption of respondents being identical (required for 

multinomial logit estimation), replacing it with a less restrictive assumption that 

respondents follow a predetermined distributional form. The theory and methods of 

mixed logit modeling are well-established (Train 2009), and it has now become standard 

practice in many areas of research (Hensher and Greene 2003). These models allow for 

coefficients on attributes to be distributed across sampled individuals according to a set 

of estimated coefficients and researcher-imposed restrictions. The model is evaluated 

numerically using random draws because choice probabilities take the form of an integral 

over a mixing distribution which does not have a closed form (Train 2009). The 

likelihood simulation for the models used 300 Halton (random) draws. 

Economic theory provides guidance regarding some, but not all, aspects of model 

specification for mixed logit models within stated preference choice experiments. For 

example, the parameter on program cost is expected to have a negative sign, reflecting a 

positive marginal utility of income. To allow for this, preliminary models included 

specifications in which the coefficient on cost was modeled as (1) fixed, (2) lognormal, 

and (3) bounded triangular. Comparison of model output suggest that superior overall 

model fit and greatest robustness of results is achieved when cost is modeled as fixed 

within the mixing distribution. Coefficients on all variables except that on program cost 

(cost) are specified as random with a normal distribution. Similar results and WTP 

estimates are generated by all preliminary model specifications. However, the illustrated 

model leads to somewhat more conservative WTP estimates than other model variants. 
June 2012 Survey Support Document 25 

Mixed logit model statistics suggest good statistical fit across the survey versions. 

Across the models, the χ

2 values ranged from 389.97 to 989.68 (all with d.f. = 21, 

p<0.0001) and pseudo R2 ranged from 0.22 to 0.29. See Exhibits II-5 to II-9 for detail on 

the significance of the model estimated for each survey version. Direct comparisons of 

statistical fit measures across different choice experiments in the literature can be 

misleading and should be viewed with extreme caution. Many measures of model fit are 

not directly comparable across different datasets or models. Nonetheless, the overall 

statistical fit of the model appears broadly similar to choice experiments found in the 

published literature addressing environmental improvements both worldwide and in the 

United States Johnston et al. (2011a,b), in a similar survey of ecological improvements, 

report a χ

2

 of 533.62 (d.f. = 12, p<0.0001) and a pseudo R2 of 0.30. By way of 

comparison using a commonly reported measure of model fit (pseudo or McFadden R2), 

Campbell et al. (2009) report a pseudo R2

 of 0.20; Carlsson et al. (2003) report pseudo 

R2 values between 0.12 and 0.27; Do and Bennett (2009) report pseudo R2 between 0.07 

and 0.18; and Colombo and Hanley (2008) report values between 0.16 and 0.36. Other 

measures of fit are also similar, although again, caution must be exercised when drawing 

conclusions from any such comparisons across models. 

Model results for all five surveys are presented in Exhibits II-5 to II-9. The 

variable for fish saved (fish_sav) is significant in all five models, commercial fish 

populations (com_fish) is significant in four of the five models, fish populations (all fish) 

(fish_pop) is significant in three of the five models, and aquatic ecological condition 

(aquatic) is statistically significant in two of the five models. The significance of these 

attributes suggests positive implicit prices, that is, positive WTP for changes in individual 
June 2012 Survey Support Document 26 

attributes. Analogous outcomes are common in choice experiments across the literature 

addressing aquatic ecological improvements, with the substantial majority of choice 

attributes found to have statistically significant impacts (e.g., Johnston et al. 2011a,b; 

Carlsson et al. 2003; Do and Bennett 2009). The alternative specific constant (ASC) 

(constant) is a fixed coefficient estimated within choice experiments, which is designed 

to capture “systematic but unobserved information about why respondents chose a 

particular option; that is, unrelated to choice set attributes” (Bennett, Rolfe and Morrison 

2001). The ASC was not significant in any of the five models, which should be viewed as 

a desirable result. ASCs become statistically significant in choice experiment models 

only when elements other than the independent variables, or choice attributes, in the 

model influence respondents’ choices (Kerr and Sharp 2006). At the time this supporting 

material was prepared, EPA had not completed the non-response studies for the 

Southeast, Pacific, Inland and National survey versions. The models summarized in 

Exhibits II-5 to II-9 do not include any weighting based on non-response studies. 

As noted above, all variables, except cost, represent percent progress towards the 

upper ecological reference condition (100%). Hence, these coefficients may be directly 

interpreted as the relative marginal utility derived from a one percentage point change in 

each ecological attribute. In the estimated Northeast model, for example, marginal utility 

is greatest (per percentage point change) for increases in aquatic ecological condition 

(aquatic), with lower, but still statistically significant. impacts associated with changes in 

commercial fish populations (fish_pop) and the number of fish saved (fish_sav). As noted 

above, the percentage differences across the options presented were much larger for the 

number of fish saved (fish_sav) than for the other variables. Following recommended 
June 2012 Survey Support Document 27 

practice in stated preference valuation, these variations correspond with realistic 

ecological and policy expectations for regulatory outcomes (Bateman et al. 2002). 

EPA notes that these model results are preliminary. EPA intends to complete the 

non-response studies for the other regions to estimate weighted models that correct for 

non-response bias. EPA also intends to apply a number of validity tests to examine the 

robustness and theoretical consistency of estimated WTP equations after the nonresponse studies have been completed for all versions. When these efforts are completed, 

EPA intends to post these findings on its website,6 and subject the complete survey 

results and analysis, including scope and validity testing, to external peer review. EPA is 

asking for public comments on the survey and results, and their use in estimating 

environmental benefits under regulatory options. 

 

6

 http://water.epa.gov/lawsregs/lawsguidance/cwa/316b/ 
June 2012 Survey Support Document 28 

Exhibit II-5—Model Results for the Northeast Region a

Variable Variable Definition Coefficient b Standard 

Error P-value 

Random parameters in utility functions 

CONSTANT ASC associated with the status quo, or choice of 

neither plan.

-0.08269 0.36139 0.8190 

COM_FISH Score showing the overall health of commercial and 

recreational fish populations.

0.20338 0.05351 0.0001 

FISH_POP Score showing the estimated size of all fish 

populations compared to natural levels without human 

influence.

0.07493 0.08347 0.3693 

FISH_SAV Score showing the reduction in young fish lost 

compared to current levels.

0.03044 0.00519 0.0000 

AQUATIC Score showing the ecological condition of affected 

areas, compared to the most natural waters in the 

Northeast.

0.20973 0.08991 0.0197 

Nonrandom parameters in utility functions

COST The increase in annual household cost, in unavoidable 

price increases for products and services, including 

electricity and common household products.

-0.02792 0.00447 0.0000 

Derived standard deviations for parameter distributions

sdCONSTANT - 0.03823 1.05647 0.9711 

sdCOM_FISH - 0.17539 0.28451 0.5376 

sdFISH_POP - 0.17880 0.31559 0.5710 

sdFISH_SAV - 0.05615 0.01159 0.0000 

sdAQUATIC - 0.26680 0.33937 0.4318 

a

 The mixed logit results have a χ

2

 of 518.40 (d.f. = 21, p<0.0001) and a pseudo R2of 0.22. b

 For random parameters in utility functions, coefficients represent the estimated means of random parameter 

distributions. 
June 2012 Survey Support Document 29 

Exhibit II-6—Model Results for the Southeast Region a

Variable Variable Definition Coefficient b Standard 

Error P-value 

Random parameters in utility functions 

CONSTANT ASC associated with the status quo, or choice of 

neither plan.

0.08004 0.34193 0.8149 

COM_FISH Score showing the overall health of commercial and 

recreational fish populations.

0.12104 0.04574 0.0081 

FISH_POP Score showing the estimated size of all fish populations 

compared to natural levels without human influence.

0.12531 0.07138 0.0791 

FISH_SAV Score showing the reduction in young fish lost 

compared to current levels.

0.02568 0.00536 0.0000 

AQUATIC Score showing the ecological condition of affected 

areas, compared to the most natural waters in the 

Southeast.

0.23161 0.07885 0.0033 

Non-random parameters in utility functions

COST The increase in annual household cost, in unavoidable 

price increases for products and services, including 

electricity and common household products.

-0.03487 0.00351 0.0000 

Derived standard deviations for parameter distributions

sdCONSTANT - 0.02489 1.16784 0.9830 

sdCOM_FISH - 0.09525 0.16377 0.5608 

sdFISH_POP - 0.09301 0.13918 0.5040 

sdFISH_SAV - 0.06804 0.00952 0.0000 

sdAQUATIC - 0.12186 0.12918 0.3455 

a

 The mixed logit results have a χ

2

 of 683.29 (d.f. = 21, p<0.0001) and a pseudo R2 of 0.24. b

 For random parameters in utility functions, coefficients represent the estimated means of random parameter 

distributions. 
June 2012 Survey Support Document 30 

Exhibit II-7—Model Results for the Pacific Region a

Variable Variable Definition Coefficient b Standard 

Error P-value 

Random parameters in utility functions 

CONSTANT ASC associated with the status quo, or choice of 

neither plan.

0.24593 0.54100 0.6494 

COM_FISH Score showing the overall health of commercial and 

recreational fish populations.

0.10278 0.10424 0.3241 

FISH_POP Score showing the estimated size of all fish 

populations compared to natural levels without 

human influence.

0.14215 0.15130 0.3475 

FISH_SAV Score showing the reduction in young fish lost 

compared to current levels.

0.05323 0.01141 0.0000 

AQUATIC Score showing the ecological condition of affected 

areas, compared to the most natural waters in the 

Pacific.

0.20853 0.16636 0.2100 

Non-random parameters in utility functions

COST The increase in annual household cost, in 

unavoidable price increases for products and 

services, including electricity and common 

household products.

-0.02226 0.00483 0.0000 

Derived standard deviations for parameter distributions

sdCONSTANT - 0.02585 1.92718 0.9893 

sdCOM_FISH - 0.20354 0.15901 0.2005 

sdFISH_POP - 0.14019 0.36490 0.7008 

sdFISH_SAV - 0.13881 0.05278 0.0085 

sdAQUATIC - 0.18715 0.50826 0.7127 

a

 The mixed logit results have a χ

2

 of 455.32 (d.f. = 21, p<0.0001) and a pseudo R2 of 0.29. b

 For random parameters in utility functions, coefficients represent the estimated means of random parameter 

distributions. 
June 2012 Survey Support Document 31 

Exhibit II-8—Model Results for the Inland Region a

Variable Variable Definition Coefficient b Standard 

Error P-value 

Random parameters in utility functions 

CONSTANT ASC associated with the status quo, or choice of 

neither plan.

-0.25032 0.35323 0.4785 

COM_FISH Score showing the overall health of commercial and 

recreational fish populations.

0.09898 0.03205 0.0020 

FISH_POP Score showing the estimated size of all fish 

populations compared to natural levels without human 

influence.

0.11198 0.05370 0.0370 

FISH_SAV Score showing the reduction in young fish lost 

compared to current levels.

0.02470 0.00427 0.0000 

AQUATIC Score showing the ecological condition of affected 

areas, compared to the most natural waters in the 

Inland region.

0.08669 0.06599 0.1890 

Non-random parameters in utility functions

COST The increase in annual household cost, in unavoidable 

price increases for products and services, including 

electricity and common household products.

-0.03186 0.00272 0.0000 

Derived standard deviations for parameter distributions

sdCONSTANT - 0.02650 2.01575 0.9895 

sdCOM_FISH - 0.07322 0.16944 0.6656 

sdFISH_POP - 0.06925 0.20775 0.7389 

sdFISH_SAV - 0.05182 0.00773 0.0000 

sdAQUATIC - 0.52674 0.40309 0.1913 

a

 The mixed logit results have a χ

2

 of 989.68 (d.f. = 21, p<0.0001) and a pseudo R2 of 0.22. b

 For random parameters in utility functions, coefficients represent the estimated means of random parameter 

distributions. 
June 2012 Survey Support Document 32 

Exhibit II-9—Model Results for the National Survey a

Variable Variable Definition Coefficient b Standard 

Error P-value 

Random parameters in utility functions 

CONSTANT ASC associated with the status quo, or choice of 

neither plan.

-0.06065 0.60390 0.9200 

COM_FISH Score showing the overall health of commercial and 

recreational fish populations.

0.17407 0.06579 0.0082 

FISH_POP Score showing the estimated size of all fish 

populations compared to natural levels without 

human influence.

0.25141 0.12384 0.0423 

FISH_SAV Score showing the reduction in young fish lost 

compared to current levels.

0.03938 0.01084 0.0003 

AQUATIC Score showing the ecological condition of affected 

areas, compared to the most natural waters in the 

U.S.

0.14043 0.13630 0.3029 

Non-random parameters in utility functions

COST The increase in annual household cost, in 

unavoidable price increases for products and 

services, including electricity and common 

household products.

-0.03539 0.00461 0.0000 

Derived standard deviations for parameter distributions

sdCONSTANT - 0.02280 1.82792 0.9900 

sdCOM_FISH - 0.13724 0.52054 0.7921 

sdFISH_POP - 0.11719 0.56426 0.8355 

sdFISH_SAV - 0.10766 0.05600 0.0545 

sdAQUATIC - 0.16083 1.21828 0.8950 

a

 The mixed logit results have a χ

2

 of 389.97 (d.f. = 21, p<0.0001) and a pseudo R2 of 0.24. b

 For random parameters in utility functions, coefficients represent the estimated means of random parameter 

distributions. 

E. Estimation of Willingness-to-Pay 

EPA used the results of the random utility models presented in Exhibits II-5 

through II-9 to estimate the marginal annual WTP (or implicit price) for a one percentage 

point change in each of the four environmental attributes within each survey region. This 

represents WTP per household, per year, for a one percentage point change in the 

corresponding choice model attribute. If utility is modeled as a linear function of 

attributes, implicit prices may be calculated as IPa = ߚ௔

ߚ ௡ൗ , where βa is the estimated 

coefficient on an environmental attribute (e.g., change in fish saved), and βn is the 

coefficient on program cost. Assuming a linear preference function as estimated above, 
June 2012 Survey Support Document 33 

compensating surplus (or household WTP) for any given policy option may be calculated 

as:7

WTP = ሺܫܲ௜௙_௠௢௖௦௛ כ∆ ܿ݋݉_݂݅ݏ݄ሻ ൅ ሺܫܲ௜௙௦௛_௣௢௣ כ∆ ݂݅ݏ݄_݌݋݌ሻ ൅ ሺܫܲ௜௙௦௛_௦௔௩ כ

 ,ሻ݅ܿݐܽݑݍܽ∆ כ ௖௜௧௔௨௔௤ܲܫሺ ൅ ሻݒܽݏ_݄ݏ݂݅∆

where the delta (∆) represents a change in the attribute in question. That is, total WTP for 

a policy change is calculated as the sum of the product of implicit prices and 

corresponding attribute changes. 

EPA notes that ecological systems are typically characterized by correlation 

among many processes and outcomes. In the context of impingement and entrainment 

losses, for example, a reduction in age-one-equivalent losses (fish_sav) may be correlated 

with changes in fish populations (fish_pop), aquatic ecosystem condition (aquatic), and 

commercial fish populations (com_fish). Were the stated preference survey scenarios to 

incorporate the same correlations, it would have been difficult to determine which 

attribute(s) caused respondents to choose one scenario over another. For example, if it 

were the case that large reductions in I&E losses always accompany large positive effects 

on fish populations and large positive effects on ecosystem condition and these 

correlations were embedded within survey scenarios, it would be difficult to estimate the 

relative influence of each attribute on respondents’ choices. 

The experimental design used in the stated preference survey breaks this 

correlation, allowing different survey attributes to vary independently. This enables 

different respondents to view many different possible policy outcomes, each with 

different combinations of fish_sav, fish_pop, aquatic and com_fish. While some of the 

 

7

 EPA excluded the ASC when estimating the benefits of regulatory options because there is no clear 

theoretical reason for inclusion. 
June 2012 Survey Support Document 34 

resulting scenarios might be unlikely in actual aquatic systems, they are not ecologically 

impossible. For example, the experimental design allows respondents to consider 

scenarios in which large reductions in fish losses accompany small changes in fish 

populations and aquatic condition (positive changes in fish_sav in some questions are 

also paired with no change in the population or aquatic condition metrics). Because 

attributes vary independently across the 72 different choice questions presented to 

respondents in each survey region, it is possible to estimate the unique effects of each 

attribute on individuals’ choices and therefore, values. By breaking the correlation 

between these attributes present in ecosystems, the choice experiment design allows 

estimation of the independent effect of each attribute on choices and willingness to pay. 

These attributes have almost zero correlation in the resulting experimental design. 

It is thus possible to obtain precise (i.e., efficient) estimates of each effect, without 

concerns that these estimates are confounded by correlations among the ecological 

outcomes in the survey. This allows willingness to pay for each ecological effect to be 

estimated, independent from all other effects. 8 

Because the mixed logit model includes random coefficients, EPA estimates 

implicit prices using the welfare simulation approach of Johnston and Duke (2007; 2009) 

following the framework outlined by Hensher and Greene (2003). The procedure begins 

with a parameter simulation following the parametric bootstrap of Krinsky and Robb 

(1986), with R=1,000 draws taken from the mean parameter vector and associated 

covariance matrix. For each draw, the resulting parameters are used to characterize 

 

8

 The experimental design utilized in EPA’s survey ameliorates the problem of correlation between 

estimated model attributes. The actual correlation among the ecological attributes, such as fish saved and 

aquatic condition, would be accounted for in the predicted attribute changes used (in conjunction with the 

estimated model coefficients) for model prediction of willingness to pay. 
June 2012 Survey Support Document 35 

asymptotically normal empirical densities for fixed and random coefficients. For each of 

these R draws, a coefficient simulation is then conducted for each random coefficient, 

with S=1,000 draws taken from simulated empirical densities. WTP measures are 

calculated for each draw, resulting in a combined empirical distribution of R×S

observations from which summary statistics are derived. All implicit prices are modeled 

as the WTP for a one percentage point change in the ecological attribute, all else being 

constant. Exhibit II-10 presents the implicit price and standard deviation for a one 

percentage point increase in each attribute. The 90 percent confidence intervals for each 

variable, based on the 5th and 95th percentiles of its empirical distribution, are also 

presented. As shown in Exhibit II-10, the estimated implicit prices are uniformly larger 

for commercial fish populations, fish populations (all fish), and aquatic ecosystem 

condition than fish saved. In the Northeast for example, households value a one 

percentage point increase in commercial fish populations or aquatic ecosystem condition 

about seven times more than a one percentage point increase in fish saved. Although the 

discussion in this section refers to willingness-to-pay for a percentage point increase in 

fish saved, it is important to note that this variable represents a one percentage point 

reduction relative to the baseline mortality (e.g., the Northeast survey booklet indicated a 

baseline loss of 1.1 billion fish). This relationship between the percentage point reduction 

and cardinal fish losses was specified clearly in survey questions, and the same 

relationship was maintained throughout each survey version. Again, using the Northeast 

survey as an example, EPA is presenting the willingness to pay for a percentage point 

reduction of mortality which is associated with a specific absolute quantity of fish out of 

1.1 billion fish, rather than a general relative reduction of one percent from an 
June 2012 Survey Support Document 36 

unspecified level of I&E mortality. The regional and national survey versions have 

different baseline fish losses. EPA expected survey responses to vary across the regions, 

both because residents might have different values, and because baseline losses differ. 

The implicit prices reflect this expected variation. EPA plans to explore whether 

respondents are more sensitive to the representation of fish saved in percentage terms, or 

in absolute numbers. 

Exhibit II-10— Estimated Implicit Prices for a One Percentage Point Change in 

Each Attribute , WTP per Household, per Year (2011$) 

Region/Attribute Implicit 

Price a

Standard 

Deviation 

90% Confidence Interval 

5th 95th

Northeast

Commercial Fish Populations (COM_FISH) $7.35 $2.15 $4.14 $11.14 

Fish Populations (all fish) (FISH_POP) $2.66 $3.13 -$2.32 $7.71 

Fish Saved (FISH_SAV) $1.12 $0.22 $0.76 $1.50 

Aquatic Ecosystem condition (AQUATIC) $7.66 $3.40 $2.35 $13.45 

Southeast

Commercial Fish Populations (COM_FISH) $3.49 $1.42 $1.30 $5.89 

Fish Populations (all fish) (FISH_POP) $3.57 $2.08 $0.27 $6.98 

Fish Saved (FISH_SAV) $0.75 $0.18 $0.46 $1.04 

Aquatic Ecosystem condition (AQUATIC) $6.61 $2.12 $3.00 $10.00 

Pacific

Commercial Fish Populations (COM_FISH) $4.67 $5.26 -$3.16 $13.35 

Fish Populations (all fish) (FISH_POP) $6.43 $7.32 -$5.12 $17.61 

Fish Saved (FISH_SAV) $2.52 $0.80 $1.39 $3.90 

Aquatic Ecosystem condition (AQUATIC) $9.50 $7.67 -$3.03 $22.38 

Inland

Commercial Fish Populations (COM_FISH) $3.10 $1.07 $1.43 $4.83 

Fish Populations (all fish) (FISH_POP) $3.48 $1.72 $0.75 $6.30 

Fish Saved (FISH_SAV) $0.78 $0.14 $0.55 $1.01 

Aquatic Ecosystem condition (AQUATIC) $2.70 $2.11 -$0.84 $6.16 

National

Commercial Fish Populations (COM_FISH) $4.93 $2.03 $1.84 $8.34 

Fish Populations (all fish) (FISH_POP) $7.04 $3.56 $1.35 $12.78 

Fish Saved (FISH_SAV) $1.13 $0.32 $0.59 $1.68 

Aquatic Ecosystem condition (AQUATIC) $3.94 $4.00 -$2.72 10.56 a

 The implicit prices are per percentage point increase from the specified baseline (reference) levels for each survey 

version. See Exhibit II-1for baseline values.
June 2012 Survey Support Document 37 

F. Method for Estimating Regional and National Benefits 

The implicit prices presented in Exhibit II-10 could be used to estimate annual 

monetized benefits for the survey regions and total U.S. households under regulatory 

options. Annual WTP per household for a regulatory option can be calculated by 

multiplying changes in environmental attributes due to the regulation (fish saved, 

commercial fish populations, fish populations (all fish), and aquatic ecosystem condition) 

by their respective implicit prices, or WTP per percentage point change. The models use 

a linear approximation for utility, so the implicit price of each attribute is constant. 

The annual household WTP under each regulatory option could be estimated 

solely based on changes in fish saved (fish_sav) by multiplying its implicit price by the 

percentage change in fish saved. EPA has calculated the marginal change in fish saved 

(%) under each regulatory option in the proposed rule, based on the percentage reduction 

in A1E losses relative to baseline A1E losses within the survey region. The resulting 

estimates of A1E reductions and percentage fish saved for each survey region and the 

United States are presented in Exhibit II-11. Changes in commercial fish populations, fish 

populations (all fish), and aquatic ecosystem condition could also factor into the 

calculation of household WTP, however, EPA has not modeled changes in these drivers 

of WTP. Within the context of the model, using only changes in fish saved would result 

in conservative estimates of benefits, in that it assumes zero impacts on all other choice 

model attributes. EPA notes that while monetized benefits based solely on fish saved 

would be conservative within the framework of the survey results and model, there are 

other reasons why the WTP estimates might be biased, as discussed previously. 
June 2012 Survey Support Document 38 

As described previously, the regions for the stated preference survey differ from 

the benefits regions used for the analysis of the proposed rule.9

 EPA estimated the A1E 

reductions in Exhibit II-11 by applying state-level data for facility actual intake flow 

(AIF) to regional A1E reductions from the proposed rule. EPA believes that the use of 

flow to extrapolate A1E reductions is appropriate for this regional and national level 

analysis, but it is important to note that the generalized relationship between flow and 

A1E described, as well as any resulting estimates of WTP for percent of fish saved, 

which are based on specific regional levels of baseline losses, may not hold at specific 

sites around the country. 

 

9

 See Exhibit II-3 for the list of states included within each survey region. 
June 2012 Survey Support Document 39 

Exhibit II-11—Reduction in A1E Losses and Percent Fish Saved by Regulatory 

Option 

Survey Version and Regulatory Option Reduction in A1E Losses Fish Saved (%) 

Northeast

Eliminating Baseline I&E Mortality Losses a 964.87 100.00

Option 1 78.31 8.12

Option 2 880.70 91.28

Option 3 893.73 92.63

Option 4 77.29 8.01

Southeast

Eliminating Baseline I&E Mortality Losses 722.97 100.00

Option 1 271.41 37.54

Option 2 642.28 88.84

Option 3 651.70 90.14

Option 4 265.86 36.77

Pacific b

Eliminating Baseline I&E Mortality Losses 385.99 100.00

Option 1 1.78 0.46

Option 2 33.28 8.62

Option 3 34.74 9.00

Option 4 1.73 0.45

Inland

Eliminating Baseline I&E Mortality Losses 462.29 100.00

Option 1 263.48 56.99

Option 2 425.29 92.00

Option 3 433.38 93.75

Option 4 257.54 55.71

National Version

Eliminating Baseline I&E Mortality Losses 2536.13 100.00

Option 1 614.97 24.25

Option 2 1981.55 78.13

Option 3 2013.55 79.39

Option 4 602.42 23.75 a This hypothetical scenario reflects the benefits that would be achieved if all I&E mortality losses were eliminated. 

EPA includes it to allow comparison of regulatory option benefits to total baseline I&E mortality losses. This scenario 

was listed as “Baseline I&E Losses” in the EEBA for the proposed rule.

b

 The calculation of Fish Saved (%) for the Pacific survey region includes reductions in A1E losses at Hawaii facilities. 

This approach is consistent with the EEBA which included Hawaii facilities in the California region. Regulatory 

Scenarios: Option 1 = IM limitations based on modified traveling screens for all facilities with flow greater than 2 

million gallons per day (MGD); Option 2 = Intake flow commensurate with closed-cycle cooling for facilities that have 

a design intake flow of greater than 125 MGD and IM limitations based on modified traveling screens for all facilities 

with flow greater than 2 MGD; Option 3 = Intake flow commensurate with closed-cycle cooling for all facilities and 

IM limitations based on modified traveling screens for all facilities with flow greater than 2 MGD; Option 4 = IM 

limitations based on modified traveling screens for all facilities with flow greater than 50 million gallons per day 

(MGD).

If the results of the stated preference survey are to be used for the final rule, EPA 

would calculate the total annual WTP within each of the survey regions under each 

regulatory option by multiplying the annual household WTP by the number of 

households in the survey region. See Exhibit II-3 for the number of households in each 
June 2012 Survey Support Document 40 

survey region. Total annual WTP within the survey region would then be discounted 

based on the regulatory compliance schedule. The compliance schedule is a time profile 

that reflects when benefits from each facility will be realized. Refer to Appendix D of the 

EEBA of the proposed rule for additional description of the compliance schedule. Since 

the compliance schedule moves the reductions in A1E into the future, this step would 

reduce the present value of the total benefits accrued to households, decreasing 

annualized benefits estimates. 

Annualized national benefits can potentially be calculated as the sum of regional 

benefits or based on the national survey version. EPA will conduct additional analysis to 

compare the results and evaluate the validity of these two approaches for estimating 

benefits to U.S. households. 

As described previously, the model estimates a constant marginal WTP (or 

implicit price) per percentage point reduction in baseline mortality, regardless of how 

much progress has already been made in reducing I&E mortality. EPA notes that this 

simplification, while common in many types of benefit analysis, does not allow one to 

quantify or test theoretically-expected effects such as diminishing marginal WTP, i.e., 

that the more of a good an individual has, the less she is willing to pay for an additional 

unit. Relative to a diminishing marginal utility form, the linear functional form likely 

produces lower estimates of WTP at low percentages of fish saved, and higher estimates 

of WTP at high percentages of fish saved. This fact could affect the relative attractiveness 

of regulatory options from a cost-benefit viewpoint; therefore, EPA will also investigate 

alternative non-linear functional forms that relax the assumption of constant utility. 
June 2012 Survey Support Document 41 

At the time this supporting material was prepared, EPA had not yet completed 

fielding the non-response studies and analyzing the non-response data for the Southeast, 

Pacific, Inland, and national versions. If EPA includes quantified benefits from the stated 

preference survey in the analysis for the final rule, EPA would update its models for the 

final rule based on the results of the non-response studies that are being conducted. EPA 

will post these results on the Agency’s website once they become available.10 The 

Northeast non-response study has been completed and a weighted model based on its 

results is discussed in the next section. EPA will also make adjustments to the estimation 

methodology that may be necessary to address issues of representativeness that arise as a 

result of the Agency’s continuing efforts at scope and validity testing, public comments, 

and comments from the planned external peer review. As already noted, EPA will 

carefully consider those comments and results before determining what role, if any, the 

quantified benefits based on this stated preference survey should play in the benefits 

analysis for the final rule. 

EPA notes that benefits estimated based on the stated preference survey would not 

be viewed as additive to those benefit values estimated at proposal. Values estimated 

using the stated preference survey would represent both use and non-use willingness-topay. As a result, if EPA includes the quantified benefits derived from the stated 

preference survey in the final benefits analysis, those benefits would supplant the 

previous benefit estimates. 

 

10 http://water.epa.gov/lawsregs/lawsguidance/cwa/316b/ 
June 2012 Survey Support Document 42 

G. Northeast Non-Response Study, Weighted Model Results, and Sensitivity to 

Scope 

As described earlier, EPA conducted the Northeast mail survey and non-response 

study in advance of the other three regional and national surveys. At the time this 

supporting material was prepared, EPA had not finished collecting and analyzing the 

non-response data for the Inland, Southeast, Pacific, and national surveys. EPA obtained 

a total of 111 completed responses for the Northeast non-response survey and has 

statistically tested the set of demographic characteristics across both the main stated 

preference survey and the non-response survey for the Northeast. Education and gender 

values were found to be statistically different. As a next step in the assessment process, 

EPA estimated regressions which included interaction variables designed to indicate 

whether both education and gender had a statistically significant impact on the estimated 

implicit prices. Based on the above results, EPA developed weights that adjust for the 

difference between the education and gender attributes across the two Northeast survey 

samples and their potential influence on the estimated implicit prices. The results of the 

weighted model are presented in Exhibit II-12. 

 EPA used the results of the weighted Northeast random utility model to estimate 

the marginal annual WTP (or implicit price) for a one percentage change in each of the 

four environmental attributes within the Northeast U.S. The resulting implicit prices are 

presented in Exhibit II-13. For the Northeast region, the implicit prices are higher when 

using the weighted results. This increase is the net result of increases and decreases from 

weighting to decrease the impact of overrepresented groups and increase the impact of 

underrepresented groups. For additional information on the non-response survey 
June 2012 Survey Support Document 43 

assessment, see EPA’s memorandum to the 316(b) rulemaking record (DCN-4524. EPA 

will analyze the non-response data for Southeast, Pacific, Inland, and national surveys 

using an approach consistent with that of the Northeast non-response data, and notes that 

the relationship between unweighted and weighted results in the Northeast region may 

not apply to the other survey versions. The implicit prices presented in Exhibit II-13 

could be used to calculate annual WTP for Northeast households under regulatory options 

following the approach described in the Section F. 

Exhibit II-12—Weighted Model Results for the Northeast Region a

Variable Variable Definition Coefficientb Standard 

Error P-value 

Random parameters in utility functions 

CONSTANT ASC associated with the status quo, or choice of 

neither plan.

-0.19167 0.34295 0.5762 

COM_FISH Score showing the overall health of commercial and 

recreational fish populations.

0.14850 0.04921 0.0025 

FISH_POP Score showing the estimated size of all fish 

populations compared to natural levels without 

human influence.

0.09356 0.07931 0.2381 

FISH_SAV Score showing the reduction in young fish lost 

compared to current levels.

0.02712 0.00548 0.0000 

AQUATIC Score showing the ecological condition of affected 

areas, compared to the most natural waters in the 

Northeast.

0.18299 0.09398 0.0515 

Non-random parameters in utility functions

COST The increase in household cost, in unavoidable price 

increases for products and services, including 

electricity and common household products.

-0.02017 0.00393 0.0000 

Derived standard deviations for parameter distributions

sdCONSTANT - 0.03763 0.93582 0.9679 

sdCOM_FISH - 0.17587 0.14002 0.2091 

sdFISH_POP - 0.18098 0.21738 0.4051 

sdFISH_SAV - 0.05255 0.01532 0.0006 

sdAQUATIC - 0.24411 0.43106 0.5712 

a

 The mixed logit results have a χ

2

 of 478.67 (d.f. = 21, p<0.0001) and a pseudo R2 of 0.22. b

 For random parameters in utility functions, coefficients represent the estimated means of random parameter 

distributions. 
June 2012 Survey Support Document 44 

Exhibit II-13— Estimated Implicit Prices for a One Percentage Point Change in 

Each Attribute in the Northeast, WTP per Household, per Year based on the 

Weighted Model (2011$) 

Attribute Implicit 

Price a

Standard 

Deviation 

90% Confidence Interval 

5th 95th

Commercial Fish Populations (COM_FISH) $7.55 $3.01 $3.37 $12.76

Fish Populations (all fish) (FISH_POP) $4.75 $4.32 -$1.84 $11.62

Fish Saved (FISH_SAV) $1.40 $0.38 $0.85 $2.08

Aquatic Ecosystem condition (AQUATIC) $9.34 $5.03 $1.35 $18.14 a

 The implicit prices are per percentage point increase from the specified baseline (reference) levels for each survey 

version. See Exhibit II-1 for baseline values.

The role of external scope tests within choice modeling has been given relatively 

little attention in the literature (cf., Heberlein et al. 2005). Unlike open-ended contingent 

valuation questions, choice experiments provide a direct mechanism for respondents to 

react to the scope and scale of resource changes, by enabling respondents to compare 

policy options with different levels for each attribute. Hence, as noted by Bennett and 

Blamey (2001, p. 231), “internal scope tests are automatically available from the results 

of a [choice modeling] exercise.” Within a choice modeling context, external scope tests 

may also be confounded by differences in the implied choice frame (Bennett and Blamey 

2001). These caveats aside, an external scope test can provide some insight into response 

patterns, and many view these tests as a “stronger” form of validation than internal scope 

tests. Hence EPA implemented some preliminary external scope tests to evaluate this 

form of validity using the Northeast survey data. 

EPA performed a split-sample external validity test for the Northeast mail survey 

data based on the concept that, if all else is orthogonal (effectively equal), a choice option 

with a greater number of fish saved should be chosen more often than a choice option 

with a lesser number of fish saved. To distinguish this from the “internal” scope tests 

automatically performed by choice experiments, it is implemented using a split sample of 
June 2012 Survey Support Document 45 

choice options viewed in isolation. To implement the test, we first create a dataset only of 

observations on Option A for all survey responses, along with the dummy (0-1) variable 

choice, indicating whether that option was chosen. We then further split this sample into 

three sub-samples: (1) observations on Option A when fish_sav = 95%, (2) observations 

on Option A when fish_sav = 50%, and (3) observations on Option A when fish_sav 

=5%. Because of the near orthogonal nature of the experimental design, all other 

attribute levels should be approximately equal across each of these three sub-samples. 

Given this split sample, we would expect to observe the greatest proportion of 

respondents choosing Option A in sub-sample (1), followed by sub-sample (2) and then 

(3). This order would establish external sensitivity to scope. We then repeated the same 

test for Option B. 

EPA found that the proportion of respondents choosing Option A (choice) 

declines from 0.45 to 0.42 to 0.25 as the percentage of fish saved (fish_sav) declines from 

95% to 50% to 5%. Option B exhibits a similar decline in choice with fish_sav. This 

external scope tests for split samples of both Options A and B demonstrate scope 

sensitivity, as indicated by economic theory. For additional information on this external 

scope test see EPA’s memorandum to the 316(b) rulemaking record (DCN-4524).