1

SUPPORTING STATEMENT 

Request to Conduct Focus Groups 

§316(b) Benefits Survey 

June 17, 2010 

(1) Title of the Information Collection 

Estimating the Social Benefits of §316(b) Requirements for Cooling Water Intake 

Structures under the National Pollutant Discharge Elimination System Permit Program 

(2) Short Characterization/Abstract 

On February 16, 2004, the U.S. Environmental Protection Agency (EPA) took final action 

on the Phase II rule governing cooling water intake structures at existing facilities that are 

point sources; that, as their primary activity, both generate and transmit electric power or 

generate electric power for sale to another entity for transmission; that use or propose to 

use cooling water intake structures with a total design intake flow of 50 MGD or more to 

withdraw cooling water from waters of the United States; and that use at least 25 percent 

of the withdrawn water exclusively for cooling purposes. See 69 FR 41576 (July 9, 

2004). Industry and environmental stakeholders challenged the Phase II regulations. On 

judicial review, the Second Circuit (Riverkeeper, Inc. v. EPA, 475 F.3d 83, (2d Cir., 

2007)) remanded several provisions of the Phase II rule. Some key provisions remanded 

are as follows: EPA improperly used a cost-benefit analysis as a criterion for determining 

Best Technology Available (BTA), and EPA inappropriately used ranges in setting 

performance expectations. In response, EPA suspended the Phase II regulation in July 

2007 pending further rulemaking. The U.S. Supreme Court granted Entergy 

Corporation’s petition for writ of certiorari, solely on the question of whether EPA had 

the authority under §316(b) of the Clean Water Act to consider costs and benefits in 

decision-making. On April 1, 2009, the Court, in Entergy Corp. v. Riverkeeper Inc., 

decided that “EPA permissibly relied on cost-benefit analysis in setting the national 

performance standards ... as part of the Phase II regulations.” EPA is now taking a 

voluntary remand of the rule, thus ending Second Circuit review. 

In June of 2006, EPA promulgated the 316(b) Phase III Rule for existing manufacturers, 

small flow power plants (facilities that use cooling water intake structures with a total 

design intake flow of less than 50 MGD to withdraw cooling water from waters of the 

United States; and that use at least 25 percent of the withdrawn water exclusively for 

cooling purposes), and new offshore oil and gas facilities. Offshore oil and gas firms and 

environmental groups petitioned for judicial review, which was to occur in the Fifth 

Circuit, but was stayed pending the Supreme Court decision on the Phase II case. EPA is 

also voluntarily remanding the existing facilities portion of the Phase III rulemaking. 

 

EPA is now combining the two phases (Phase II and III) into one rulemaking covering all 

existing facilities. EPA will develop regulations to provide national performance 
2

standards for controlling impacts from Phase II and Phase III cooling water intake 

structures under Section 316(b) of the Clean Water Act (CWA). 

Under Executive Order 12866, EPA is required to estimate the potential benefits and 

costs to society. The purpose of this set of focus groups is to finish the development of a 

survey instrument that will be used to elicit values for changes in fish mortality and the 

associated ecological gains from a section 316(b) regulation covering Phase II and Phase 

III facilities. 

The survey will be used to estimate total values (willingness to pay, or WTP) derived by 

average households for changes related to the reduction of fish losses at CWIS. As 

indicated in the prior literature (Cummings and Harrison 1995; Johnston et al. 2003, 

2005), it is virtually impossible to justify, theoretically, the decomposition of empirical 

estimates of use and nonuse values. The survey will provide the flexibility to estimate 

nonuser values, using various nonuser definitions drawn from responses to survey 

questions 11 to 13. The structure of choice attribute questions will also allow the analysis 

to separate value components related to the most common sources of use values—effect 

on harvested recreational and commercial fish. The remaining welfare estimates are 

expected to be composed primarily of nonuse components, although some use value 

components might also be included for some respondents. In summary, the survey will 

provide estimates of total values, will allow estimates of value associated with specific 

choice attributes (following standard methods for choice experiments), and will also 

allow the flexibility to provide some insight into the relative importance of use versus 

nonuse values in the 316(b) context. 

Within rulemaking, among the most crucial concerns is the avoidance of benefit (or cost) 

double counting. Here, for example, the WTP estimates will include use and nonuse 

values among a representative population sample. These will overlap—to a potentially 

substantial extent—with use value estimates that might be provided through some other 

methods, including revealed preference methods that might be used to estimate use 

values of recreational anglers for fish kill reductions (i.e., through related improvements 

in fishing quality). While using the proposed stated preference value estimates for 

benefit estimation, particular care will be given to avoid any possible double counting of 

values that might be derived from alternative valuation methods. In doing so, the Office 

of Water will rely upon standard theoretical tools for non-market welfare analysis, as 

presented by authors including Freeman (2003) and Just et al. (2004). From a purely 

mechanistic perspective, survey results will be used to derive use and nonuse values 

following standard practice for choice experiments (Adamowicz et al. 1998). 

To obtain reliable estimates of WTP, the Agnecy will test and design all survey elements 

to promote incentive compatible preference elicitation mechanisms. Incentive compatible 

stated preference surveys provide no incentive for non-truthful preference revelation 

(Carson and Groves 2007). The literature is clear regarding the importance of incentive 

compatibility in stated preference value elicitation and the role of both question format 

and scenario consequentiality in ensuring this property (Carson et al. 2000; Carson and 

Groves 2007; Collins and Vossler 2009; Herriges et al. 2010; Johnston 2006; Vossler and 
3

Evans 2009). It has been established that referendum-type stated preference choices are 

incentive compatible given that certain conditions are met, including the condition that 

responses are believed by respondents to be consequential, or potentially influencing 

public policy decisions (Carson and Groves 2007; Herriges et al. 2010). Additional 

requirements for consequentiality in choice experiment surveys are detailed by Collins 

and Vossler (2009), who find that common choice experiment (or conjoint) formats are 

similarly incentive compatible under a broad range of assumptions, including those 

underlying the proposed survey instrument.

This supporting statement addresses the testing of a draft survey instrument. 

(3) Need for the Collection 

The project is being undertaken pursuant to section 104 of the Clean Water Act dealing 

with research. Section 104 of the Clean Water Act authorizes and directs the EPA 

administrator to conduct research into a number of subject areas related to water quality, 

water pollution, and water pollution prevention and abatement. This section also 

authorizes the EPA administrator to conduct research into methods of analyzing the costs 

and benefits of programs carried out under the Clean Water Act. 

This project is exploring how public values for fishery resources (covering both use and 

non-use values) are affected by fish losses from impingement and entrainment at cooling 

water intake structures (CWIS). Understanding total public values, including the more 

difficult to estimate non-use values, for fishery resources is necessary to determine the 

full range of benefits associated with reductions in impingement and entrainment losses, 

and whether the benefits of government action to reduce impingement and entrainment 

losses at Phase II and Phase III facilities are commensurate with the costs associated with 

such actions. Because non-use values may be substantial, failure to recognize such values 

may lead to improper inferences regarding policy benefits and costs. The findings from 

this study will primarily be used by EPA to improve estimates of the economic benefits 

of the section 316(b) regulation for Phases II and III facilities as required under Executive 

Order 12866. 

(4) Non-duplication 

EPA has not identified any previous studies that would allow direct estimation of the 

social benefits of quantified nationwide changes in populations of fish species (including 

forage, recreational, and commercial species) affected by the proposed 316(b) regulation. 

Therefore, to the best of our knowledge this survey is unique and does not duplicate other 

efforts. 

There are many studies in the environmental economics literature that quantify benefits 

or willingness to pay (WTP) associated with various types of water quality and aquatic 

habitat changes. However, none of these studies allows the isolation of non-market WTP 

associated with quantified reductions in fish losses (or increases in fish populations) for 

forage fish. Most available studies estimate WTP for broader, and sometimes 

ambiguously defined, policies that simultaneously influence many different aspects of 
4

aquatic environmental quality and ecosystem services, but for which WTP associated 

with fish or aquatic life alone cannot be identified. Other studies provide benefit 

estimates associated with improvements in fish (or aquatic) habitat, but do not link this to 

well-defined and quantified changes in affected or supported organisms. Still other 

studies address willingness to pay for changes in charismatic or recreational species that 

have little relationship to the types of forage fish that are the main species affected by 

cooling water intake structures. 

For example, choice experiment studies such as Hanley et al. (2006a, b) and Morrison 

and Bennett (2004) estimate WTP for aquatic ecosystem changes that affect fish, but 

effects on fish are quantified and valued solely in terms of the presence/absence of 

different types of fish species. This approach renders associated results unsuitable for 

316(b) benefit estimation. Also, many of these studies were conducted outside the U.S. 

(e.g., the EU or Australia), making their use for benefit transfer to a U.S. policy context 

much more challenging. 

Other studies have estimated the value of changes in catch rates or populations of select 

recreational and commercial species, charismatic species such as salmon, or changes in 

water quality that affect fish, but none have specifically valued changes in forage fish 

populations. For example, Olsen et al. (1991) conducted a survey of Pacific Northwest 

residents, including both anglers and non-anglers, to determine their WTP for doubling 

the size of the Columbia River Basin salmon and steelhead runs. EPA’s proposed survey 

approach differs from this study and others like it (such as Cameron and Huppert, 1989) 

in that it would include respondents from various geographic regions in the United States 

and would provide values for the full range of forage, recreational, and commercial 

species affected by 316(b) regulations, instead of valuing a few recreational species in 

one specific geographical area. 

Among available studies, the most closely related is Johnston et al. (2009), which 

estimates total willingness to pay (WTP) for multi-attribute aquatic ecosystem changes 

related to improvements in small migratory (diadromous) fish. Unlike other studies, the 

choice experiment data of Johnston et al. (2009) allow estimation of WTP associated with 

quantified changes in forage fish (e.g., WTP per fish or percentage change in fish), 

holding other ecological effects constant. That is, unlike results provided by other studies 

in the literature, WTP estimates of Johnston et al. (2009) are not confounded with values 

for other changes including water quality, habitat, overall ecological condition, etc. In 

addition, the choice experiment of Johnston et al. (2009) addresses species such as 

alewife and blueback herring that are neither subject to recreational or commercial 

harvest in Rhode Island, nor are charismatic species. Hence, the species affected are a 

close analog to the forage fish that are at primary issue in the 316(b) policy context. 

Although the methods and data of Johnston et al. (2009) allow quantification of 

combined use and nonuse values associated with specific improvements in forage and/or 

recreational fish, and also provide limited ability to isolate nonuse values from use 

values, the policy context and scale prevent direct use for analysis of national benefits of 

the 316(b) regulation. Specifically, Johnston et al. (2009) estimate Rhode Island 
5

residents’ preferences for the restoration of migratory fish passage over dams in the 

Pawtuxet and Wood-Pawcatuck watersheds. Hence, the case study is for a watershedlevel policy with statewide welfare implications. In contrast, 316(b) policies would have 

nationwide implications, both on ecosystems and on affected facilities. 

(5) Consultations 

The Principal Investigator for the stated-preference portion of this effort is Dr. Robert 

Johnston. Dr. Johnston is assisted by Dr. Elena Besedin, a Senior Economist at Abt 

Associates Inc. Dr. Erik Helm at the U.S. Environmental Protection Agency serves as the 

project manager and a contributor to this research. 

Robert J. Johnston is Director of the George Perkins Marsh Institute and Professor of 

Economics at Clark University. He is President-elect of the Northeastern Agricultural 

and Resource Economics Association (NAREA), on the Program Committee for the 

Charles Darwin Foundation, the Science Advisory Board for the Communication 

Partnership for Science and the Sea (COMPASS), and is the Vice President of the Marine 

Resource Economics Foundation. Professor Johnston has published extensively on the 

valuation of non-market commodities (goods, services, and resources), benefit cost 

analysis, and resource management. His recent research emphasizes coordination of 

ecological and economic models to estimate ecosystem service values, with particular 

emphasis on the role of aquatic ecological indicators. He has also worked extensively in 

methodologies for benefit transfer, including the use of meta-analysis. Professor 

Johnston’s empirical work on non-market valuation and benefit transfer has contributed 

to numerous benefit cost analyses conducted by federal, state and local government 

agencies in the US, Canada and elsewhere. 

Elena Y. Besedin, a senior economist at Abt Asspciates Inc., specializes in the economic 

analysis of environmental policy and regulatory programs, Her work for EPA has 

concentrated on analyzing economic benefits from reducing risks to the environment and 

human health and assessing environmental impacts of regulatory programs for many EPA 

program offices. She worked extensively on valuation of non-market benefits associated 

with environmental improvements of aquatic resources. Dr. Besedin’s empirical work on 

non-market valuation included design and implementation of stated and revealed 

preference studies and benefit transfer methodologies. Her recent work focused on 

developing integrated frameworks to value changes in ecosystem services stemming from 

environmental regulations. 

EPA has conducted an external peer review of the previous version of the survey 

instrument. The previous version of the survey instrument was reviewed by Dr. Richard 

C. Bishop, Dr. Kevin J. Boyle, Dr. Timothy C. Haab, Dr. James R. Kahn, and Dr. Alan J. 

Krupnick. The review focused on (1) the appropriateness of the survey design for 

measuring fish resources affected by cooling water intake structures (CWIS), (2) 

sufficiency and appropriateness of information provided in the survey on the baseline 

ecological conditions, the expected changes, and the expected costs, and (3)whether the 

answers to this survey lend themselves to straightforward economic interpretation of 
6

WTP. The reviewers acknowledged the extensive effort that was placed into survey 

design and noted that the 316(b) survey was a well-designed study. They also offered 

concrete suggestions for further improvements of the survey instrument. EPA has 

incorporated many of the reviewers comments in subsequent survey revisions (Versar 

2006). 

(6) Peer Review Plans 

Internal Agency reviewers provided and will continue to provide input on the content and 

format of the focus groups, questionnaire design issues, and issues related to survey 

sampling design and methodology. Survey materials will be subject to routine internal 

review by Dr. Helm and EPA’s National Center for Environmental Economics (NCEE) 

staff. 

We note that the current survey instrument is built upon an earlier version that was peer 

reviewed in January 2006. It incorporates recommendations received from the first peer 

review panel. Because the final product of this study meets the major technical work 

criteria specified in the Peer Review Handbook (U.S. EPA 2000) the Agency also plans 

to convene a peer-review panel to review the entire survey process. First, the Agency 

plans to review the survey instrument prior to fielding it. After the survey is completed, 

EPA will peer-review study results, and EPA’s final estimated results for the 316(b) 

Existing Facilities rulemaking, 

(7) Confidentiality 

The survey instrument will fully conform to federal regulations – specifically the Privacy 

Act of 1974 (5 U.S.C. 552a), the Hawkins-Stafford Amendments of 1988 (P.L 100-297), 

and the Computer Security Act of 1987. Each prospective respondent will be informed 

that their participation in the survey is voluntary, and that their identities will be kept 

confidential by the investigators and not associated with their responses. Neither EPA 

nor any other federal or state agency will have access to the names of respondents. 

The focus groups will be recorded using audiotapes, which will later be transcribed. 

However, individuals will not be identified on the transcripts. To insure that the focus 

groups include a representative and diverse sample of individuals, EPA will tabulate 

basic demographic information for participants, such as age, ethnicity, occupation, and 

income. However, no personal identifying information about participants, such as names, 

phone numbers, or addresses, will be included in the focus group transcripts. Prior to 

commencing any audio recordings, respondents would be informed of EPA's intentions to 

audiotape the focus group proceedings, and informed that all taped responses will be 

strictly anonymous and confidential. 

(8) Sensitive Questions 
7

There are no questions on sexual behavior and attitudes, religious beliefs, and other 

matters that are commonly considered private or sensitive in these focus group materials. 

Answers to standard demographic questions (i.e., age, income) will be included. 

(9) Respondents 

EPA intends to conduct eight focus groups as part of the testing of the survey instrument. 

There are a total of seven study regions for purposes of evaluating the 316(b benefits. For 

the purpose of the stated preference survey implementation, EPA uses five geographic 

regions: California, Great Lakes, Inland, Northeast, and Southeast. The Northeast region 

includes the North Atlantic and Mid Atlantic regions and the Southeast region includes 

the South Atlantic and Gulf of Mexico regions. Because focus groups in different study 

regions might provide distinct information relevant to survey design the Agency will 

conduct focus groups in multiple regions. The Agency will modify the survey instrument 

for each region relative to the current survey draft for the Northeast region as follows: 

• Cover – The text on the cover currently reads “A survey of Northeast Residents 

(CT, DE, MA, MD, ME, NH, NJ, NY, RI, VT)”. “Northeast Residents” and the 

list of included states will be changed to match the respondent’s region. 

• Cover – The photo on the cover is of a forage species (silversides) in found in the 

Northeast region. If that species isn't found in other regions, we will replace the 

cover photo with a similar substitute photo for a forage species relevant to that 

region. 

• Page 1 – The survey states that it “asks for your opinions regarding policies that 

would affect fish and habitat in the Northeast US.” “Northeast US” will be 

replaced with the respondent’s region. 

• Page 1 – Includes the statement that “Northeast fresh and salt waters support 

billions of fish.” “Northeast” will be replaced with the respondent’s region and 

“salt water” will be removed from the Inland and Great Lakes regions. 

• Page 2 - The survey states that “Cooling water use affects fresh and salt waters 

throughout the Northeast US, but almost all fish losses are in salt waters such as 

coastal bays.” For the California and Southeast regions, which include both salt 

water and freshwater facilities, reference to the Northeast region would be 

replaced with the name of the respondent’s region. For the Great Lakes and 

Inland survey regions, which only include freshwater facilities, reference to salt 

water will be removed. 

• Page 2 – A map of the Northeast region and facility locations is presented. A 

comparable map will be produced for each region, and map included in the survey 

will correspond to the respondent’s region. 

• Page 4 - This page provides an example for two Northeast coastal species and 

includes a figure comparing estimates of young fish lost to the total number of 

adult fish. The estimate of total losses and example species will be modified to 

describe species and losses occurring within the respondent’s region. 

• Page 5 - The total losses (1.1 billion) currently included within the survey 

corresponds to losses from facilities in Northeast coastal waters. This total would 

be replaced with the total losses for the respondent’s region. The pie charts will 
8

be updated based on the regional total while maintaining the 25% and 95% 

reductions used in the example. 

• Page 7 - The table text describing each score will be changed to include the 

current scores for the respondent’s region. 

• Page 8 - Text describing the commercial fish sustainability score and fish 

population score will be changed to describe the current score for the respondent’s 

region relative to its maximum. 

• Page 9 – The text states that the aquatic ecosystem score measures “how close 

affected Northeast waters are to the most natural, undisturbed condition that is 

possible.” “Northeast waters” will be replaced with a reference to waters within 

the respondent’s region. 

• Page 12 – The included figure illustrates the location of “Coastal Facilities 

Affected by the Proposed Policies”. For other regions which include both salt 

water and freshwater facilities, the figure will be replaced with a similar figure 

circling salt water facilities within the respondent’s region. For the Great Lakes 

and Inland regions (which do not include salt water facilities), only facilities 

within the region will be shown in the figure, making the circle unnecessary. 

• Pages 13-16 – Regional references within the table headings in Questions 4, 5, 

and 6 (e.g., “Policy Effect NE Coastal”, “Option A NE Coastal”) will be modified 

to refer to the policy effects and options for the respondent’s region. The “Fish 

Saved per Year” score includes a note reminding the respondent of total fish lost 

within the region due to I&E (e.g., 1.5 billion); this number will be changed to 

reflect total losses within the respondent’s region. The values describing the 

current situation and Options A and B within experiment questions 4, 5, and 6 

will also vary across regions. 

• Page 17 – Text describing “National versus Northeast policies” will be modified 

to refer to the respondent’s region. 

• Page 21 – Question 12 asks the respondent how times he participated each of six 

activity categories over the last year within the “Northeastern US only”. Text will 

be changed to reflect the respondent’s region including the list of states which the 

region includes. 

EPA will edit the Northeast survey instrument so that respondents in each of the 5 study 

regions receives a survey that is representative of their region. The specific details of the 

presentation of this region specific material is to be informed and tested by the focus 

group process. In general the instrument will be edited so that all references to 

geographic location, fish species and environmental baseline information, and regionally 

specific policy outcomes will match the study region. 

Each focus group will last approximately 90 minutes. We anticipate that eight to ten 

people will attend each focus group. For each focus group, individuals will be recruited 

from one of the study regions to reflect the range of ultimate potential respondents to a 

survey. The groups will be led by an experienced moderator, and held in facilities with 

audio/video recording capabilities. 
9

Focus group participants will be randomly selected by marketing research firms from 

panels of focus group participants maintained by each firm. Participants are eligible if 

they are 18 years of age or older; if they are not full-time students; if they do not have 

occupations related to the environment, including water treatment and sewage, electric or 

nuclear power companies; and if they have not participated in a survey or focus group in 

the last six months. Each of the focus groups will include a mix of individuals with 

diverse socioeconomic backgrounds, based on characteristics such as income, marital 

status, age, race, education, occupation, and gender. Participants will be asked to attend a 

focus group session and participate in a discussion of specific topics led by a moderator. 

Participation in the focus group sessions is voluntary. Participants will have to expend 

time, effort, and travel to participate in the focus group sessions. Following standard 

practice in marketing research, participants will be compensated for their time and effort. 

As described by Dillman (2000) in the context of population surveys, compensation can 

increase response rates and provide an incentive to participate among respondents who 

may not have a strong ex ante interest in the subject matter. Therefore, compensation can 

help to avoid self selection biases associated with ex ante interests in environmental 

policies, and can thereby encourage a more representative sample of focus group 

participants. The proposed incentive fee is $75.00 per participant. 

(10) Collection Schedule 

The proposed timeline for the data collection is as follows. 

Task: Date: 

Submit ICR package for focus groups to OMB March 2010 

Conduct focus groups June - July 2010 

Finalize draft survey instrument June 2010 

Submit ICR package to OMB July 2010 

Receive approval to conduct survey December 2010 

Field survey to sample March2011 

Allow 8-10 weeks for data collection 

Analyze data May - June 2011 

Prepare final report and papers July 2011 

Internal EPA peer review of survey methodology 

and results TBD 

(11) Respondent Burden 

For each of the eight groups we will recruit ten participants. Participants will be given 

directions to a marketing research firm where the focus group will occur. They will be 

provided a light snack and beverages as well as $75 compensation for their time. We 

anticipate that there will be some attrition between recruiting and the actual time of the 

focus group, so we anticipate eight to nine participants in each session. 

For the purposes of calculating burden, the maximum total number of respondents is 80. 

Each individual will participate in a 90 minute session. We also anticipate that 
10

individuals will spend up to 30 minutes of travel time. Therefore, the total number of 

respondent hours requested is 160 hours. 

(12) Information Requested 

Individuals who attend the focus groups will at some point during the session be asked to 

review the draft survey, answer draft survey questions, and provide feedback on these 

presentation materials and survey questions. After reviewing introductory materials and 

administering draft survey questions, the moderator of the focus group will read and ask 

participants debriefing questions. A script containing relevant questions that will be used 

to guide the discussion is attached to this request. EPA notes that the exact wording of 

debriefing questions in focus groups may be adjusted somewhat by focus group 

moderators to conform to the flow of ongoing discussion. Survey materials will be 

revised between groups based on respondent reactions and comments to ensure that the 

draft instrument used in each focus group is as clear, concise, and comprehensive as 

possible. 

EPA notes that not all discussion topics or questions presented in the attached script can 

be covered in any single focus group session, due to time constraints. Instead, these 

questions will be covered over a number of different focus groups. In some focus groups, 

the moderator will first offer some draft valuation questions to focus group participants, 

as noted above, followed by debriefing questions. In other cases, the focus group may 

begin with general discussion questions, prior to the consideration of draft survey 

questions. The order of questions either within or between the categories above may also 

be altered, as appropriate to the flow of discussion within a particular focus group. 

Freedom to alter question order is important, as it is often impossible to predict the 

direction in which respondents will direct their discussions and/or answers. 

Some of the focus groups will incorporate individual one-on-one protocol interviews of 

the survey instrument. One-on-one protocol interviews will allow in-depth exploration of 

the cognitive processes used by respondents to answer survey questions, without the 

potential for interpersonal dynamics to sway respondents’ comments (Kaplowicz et al. 

2004). 

REFERENCES 

Adamowicz, W., P. Boxall, M. Williams, and J. Louviere. 1998. Stated Preference Approaches 

for Measuring Passive Use Values: Choice Experiments and Contingent Valuation. American 

Journal of Agricultural Economics 80(1): 64-75. 

Cameron, T.A. and Huppert, D.D. (1989) OLS versus ML Estimation of Non-market Resource 

Values with Payment Card Interval Data. Journal of Environmental Economics and 

Management. 17, 230-246. 

Carson, R. T., T. Groves, and M. J. Machina. 2000. “Incentive and Informational Properties of 

Preference Questions.” Working Paper, Department of Economics, University of California, 

San Diego. 
11

Carson, R.T. and T. Groves. 2007. Incentive and Informational Properties of Preference 

Questions. Environmental and Resource Economics 37(1):181–210. 

Collins, J.P. and C.A. Vossler. 2009. Incentive Compatibility Tests of Choice Experiment Value 

Elicitation Questions. Journal of Environmental Economics and Management 58(2): 226-235. 

Cummings, Ronald G., and Harrison, Glenn W., "The Measurement and Decomposition of 

Nonuse Values: A Critical Review", Environmental and Resource Economics, 5, 1995, 225-

247. 

Freeman III, A.M. 2003. The Measurement of Environmental and Resource Values: Theory and 

Methods. Resources for the Future, Washington, DC. 

Hanley N., Colombo, S., Tinch, D., Black, A. and Aftab, A. (2006) Estimating the benefits of 

water quality improvements under the Water Framework Directive: are benefits 

transferable? European Review of Agricultural Economics 33: 391-413. 

Hanley, N., Wright, R.E. and Alvarez-Farizo, B. (2006) Estimating the economic value of 

improvements in river ecology using choice experiments: an application to the water 

framework directive. Journal of Environmental Management 78: 183-193. 

Herriges, J., C. Kling, C. Liu and J. Tobias. 2010. What are the Consequences of 

Consequentiality? Journal of Environmental Economics and Management 59(1): 67-81. 

Johnston, R.J. 2006. Is Hypothetical Bias Universal? Validating Contingent Valuation Responses 

Using a Binding Public Referendum. Journal of Environmental Economics and Management

52(1): 469-481. 

Johnston, R.J., E.Y. Besedin and R.F. Wardwell. 2003. Modeling Relationships Between Use and 

Nonuse Values for Surface Water Quality: A Meta-Analysis. Water Resources 

Research 39(12): 1363-1372. 

Johnston, R.J., E.Y. Besedin, R. Iovanna, C.J. Miller, R.F. Wardwell and M.H. Ranson. 2005. 

“Systematic Variation in Willingness to Pay for Aquatic Resource Improvements and 

Implications for Benefit Transfer: A Meta-Analysis.” Canadian Journal of Agricultural 

Economics 53(2-3):221-248. 

Johnston, R.J., E.T. Schultz, K. Segerson and E.Y. Besedin. (2009) Bioindicator-Based Stated 

Preference Valuation for Aquatic Habitat and Ecosystem Service Restoration, in Bennett, 

J. ed. International Handbook on Non-Marketed Environmental Valuation. Cheltenham, 

UK: Edward Elgar, forthcoming. 

Just, R.E., D.L. Hueth and A. Schmitz. 2004. The Welfare Economics of Public Policy: A 

Practical Approach to Public Policy. Cheltenham, UK: Edward Elgar. 

Kaplowitz, M.D., F. Lupi and J.P. Hoehn. 2004. “Multiple Methods for Developing and 

Evaluating a Stated-Choice Questionnaire to Value Wetlands.” Chapter 24 in Methods for 

Testing and Evaluating Survey Questionnaires, eds. S. Presser, J.M. Rothget, M.P. Coupter, 

J.T. Lesser, E. Martin, J. Martin, and E. Singer. New York: John Wiley and Sons. 

Morrison, M. and Bennett, J. (2004) Valuing New South Wales rivers for use in benefit transfer. 

The Australian Journal of Agricultural and Resource Economics 48: 591-611. 

Olsen, D., J. Richards, and R.D. Scott. 1991. “Existence and Sport Values for Doubling the Size 

of Columbia River Basin Salmon and Steelhead Runs.” Rivers. 2(1):44-56. 

U.S. Environmental Protection Agency (U.S. EPA). 2000. Science Policy Council Handbook: 

Peer Review. U.S. EPA,Office of Research and Development, Washington D.C., EPA 

100-B-00-001. 

Versar. 2006. Comments Summary Report: Peer Review Package for "Willingness to Pay Survey 

Instrument for §316(b) Phase III Cooling Water Intake Structures.” Prepared by Versar 

Inc., Springfield, VA. 
12

Vossler, C.A. and M.F. Evans. 2009. Bridging the Gap Between the Field and the Lab: 

Environmental Goods, Policy Maker Input and Consequentiality. Journal of Environmental 

Economics and Management 58(3): 338-345.