1 SUPPORTING STATEMENT
Request to Conduct Focus Groups

§ 316(b) Benefits Survey

June 17, 2010

(1) Title of the Information Collection

Estimating the Social Benefits of §316(b) Requirements for Cooling Water Intake Structures under the National Pollutant Discharge Elimination System Permit Program

(2) Short Characterization/Abstract

On February 16, 2004, the U.S. Environmenta l Protection Agency (EPA) took final action on the Phase II rule governing cooling water inta ke structures at existing facilities that are point sources; that, as their primary activity, both generate and transmit electric power or generate electric power for sale to another en tity for transmission; th at use or propose to use cooling water intake structures with a tota l design intake flow of 50 MGD or more to withdraw cooling water from wa ters of the United States; and that use at least 25 percent of the withdrawn water exclusively for cooling purposes. See 69 FR 41576 (July 9, 2004). Industry and environmental stakeholders challenged the Phase II regulations. On judicial review, the Second Circuit (River keeper, Inc. v. EPA, 475 F.3d 83, (2d Cir., 2007)) remanded several provisions of the Ph ase II rule. Some ke y provisions remanded are as follows: EPA improperly used a cost-ben efit analysis as a criterion for determining Best Technology Available (BTA), and EPA inappropriately used ranges in setting performance expectations. In response, EPA suspended the Phase II regulation in July 2007 pending further rulemaking. The U.S. Supreme Court granted Entergy Corporation’s petition for writ of certiorari , solely on the question of whether EPA had the authority under §316(b) of the Clean Water Act to consider costs and benefits in decision-making. On April 1, 2009, the Court, in Entergy Corp. v. Riverkeeper Inc., decided that “EPA permissibly relied on co st-benefit analysis in setting the national performance standards ... as part of the Phase II regulations.” EPA is now taking a voluntary remand of the rule, thus ending Second Circuit review.

In June of 2006, EPA promulgated the 316(b) Ph ase III Rule for existing manufacturers, small flow power plants (facilities that use cooling water intake structures with a total design intake flow of less than 50 MGD to w ithdraw cooling water from waters of the United States; and that use at least 25 percent of the withdrawn water exclusively for cooling purposes), and new offshore oil and ga s facilities. Offshore oil and gas firms and environmental groups petitioned for judicial review, which was to occur in the Fifth Circuit, but was stayed pending the Supreme Court decision on the Phase II case. EPA is also voluntarily remanding th e existing facilities portion of the Phase III rulemaking.

EPA is now combining the two phases (Phase II and III) into one ru lemaking covering all existing facilities. EPA will develop regulations to provide national performance2 standards for controlling impacts from Ph ase II and Phase III cooling water intake structures under Section 316(b) of the Clean Water Act (CWA).

Under Executive Order 12866, EPA is required to estimate the potential benefits and costs to society. The purpose of this set of fo cus groups is to finish the development of a survey instrument that will be used to elicit values for changes in fish mortality and the associated ecological gains from a section 316(b) regulation coveri ng Phase II and Phase

III facilities.

The survey will be used to estimate total values (willingness to pay, or WTP) derived by average households for changes related to th e reduction of fish losses at CWIS. As indicated in the prior lite rature (Cummings and Harrison 1995; Johnston et al. 2003, 2005), it is virtually impossible to justify, th eoretically, the decomposition of empirical

estimates of use and nonuse values. The surv ey will provide the flexibility to estimate

nonuser values, using various nonuser definitio ns drawn from responses to survey questions 11 to 13. The structure of choice at tribute questions will also allow the analysis to separate value components related to th e most common sources of use values—effect

on harvested recreational and commercial fi sh. The remaining welfare estimates are expected to be composed primarily of nonuse components, although some use value components might also be included for some respondents. In summary, the survey will provide estimates of total values, will allow estimates of value associated with specific choice attributes (following standard met hods for choice experiments), and will also allow the flexibility to provide some insight into the relative importance of use versus

nonuse values in the 316(b) context.

Within rulemaking, among the most crucial concerns is the a voidance of benefit (or cost) double counting. Here, for example, the WTP estimates will include use and nonuse values among a representative population samp le. These will overlap—to a potentially

substantial extent—with use value estimates that might be provided through some other methods, including revealed preference methods that might be used to estimate use values of recreational anglers for fish k ill reductions (i.e., through related improvements in fishing quality). While using the propos ed stated preference value estimates for benefit estimation, particular care will be gi ven to avoid any possible double counting of values that might be derived from alternativ e valuation methods. In doing so, the Office of Water will rely upon standard theoretical tools for non-market welfare analysis, as presented by authors including Freeman (2003) and Just et al. ( 2004). From a purely mechanistic perspective, survey results will be used to derive use and nonuse values following standard practice for choice experiments (Adamowicz et al. 1998).

To obtain reliable estimates of WTP, the Agn ecy will test and design all survey elements to promote incentive compatible preference el icitation mechanisms. Incentive compatible stated preference surveys provi de no incentive for non-trut hful preference revelation (Carson and Groves 2007). The literature is clear regarding the im portance of incentive compatibility in stated preference value elic itation and the role of both question format and scenario consequentiality in ensuring th is property (Carson et al. 2000; Carson and Groves 2007; Collins and Vossler 2009; Herrige s et al. 2010; Johnston 2006; Vossler and3 Evans 2009). It has been estab lished that referendum-type st ated preference choices are incentive compatible given that certain c onditions are met, incl uding the condition that responses are believed by respondents to be consequential, or potentially influencing public policy decisions (Carson and Groves 2007; Herriges et al. 2010). Additional requirements for consequentiality in choice experiment surveys are detailed by Collins and Vossler (2009), who find that common choi ce experiment (or conjoint) formats are similarly incentive compatible under a br oad range of assumptions, including those underlying the proposed survey instrument.

This supporting statement addresses the tes ting of a draft survey instrument.

(3) Need for the Collection

The project is being undertaken pursuant to section 104 of the Clean Water Act dealing

with research. Section 104 of the Clean Water Act auth orizes and directs the EPA

administrator to conduct research into a number of subject ar eas related to water quality, water pollution, and water pollution preventi on and abatement. This section also

authorizes the EPA administrato r to conduct research into me thods of analyzing the costs and benefits of programs carried out under the Clean Water Act.

This project is exploring how public values for fishery re sources (covering both use and non-use values) are affected by fish losses fr om impingement and entrainment at cooling water intake structures (CWI S). Understanding total public values, including the more difficult to estimate non-use values, for fisher y resources is necessary to determine the full range of benefits associated with reduc tions in impingement and entrainment losses, and whether the benefits of government ac tion to reduce impingement and entrainment

losses at Phase II and Phase III facilities are co mmensurate with the costs associated with such actions. Because non-use values may be s ubstantial, failure to recognize such values may lead to improper inferences regarding pol icy benefits and costs. The findings from this study will primarily be used by EPA to improve estimates of the economic benefits of the section 316(b) regulation for Phases II and III facilities as required under Executive

Order 12866.

(4) Non-duplication

EPA has not identified any pr evious studies that would a llow direct estimation of the social benefits of quantified nationwide cha nges in populations of fish species (including forage, recreational, and commercial species) affected by the proposed 316(b) regulation. Therefore, to the best of our knowledge this survey is unique and does not duplicate other

efforts.

There are many studies in the environmental eco nomics literature that quantify benefits or willingness to pay (WTP) associated with various types of water quality and aquatic habitat changes. However, none of these stud ies allows the isolat ion of non-market WTP associated with quantified reductions in fish losses (or increases in fish populations) for forage fish. Most available studies estimate WTP for broader, and sometimes ambiguously defined, policies that simultaneous ly influence many different aspects of4 aquatic environmental quality and ecosyste m services, but for which WTP associated with fish or aquatic life al one cannot be identified. Other studies provide benefit estimates associated with improvements in fish (or aquatic) habitat, but do not link this to well-defined and quantified changes in affect ed or supported organisms. Still other studies address willingness to pay for changes in charismatic or recreational species that have little relationship to the types of forage fish that ar e the main species affected by cooling water intake structures.

For example, choice experiment studies such as Hanley et al. (2006a, b) and Morrison and Bennett (2004) estimate WTP for aquatic eco system changes that affect fish, but effects on fish are quantified and valued so lely in terms of the presence/absence of different types of fish species. This appro ach renders associated results unsuitable for 316(b) benefit estimation. Also, many of thes e studies were conducted outside the U.S. (e.g., the EU or Australia), making their use fo r benefit transfer to a U.S. policy context much more challenging.

Other studies have estimated th e value of changes in catch ra tes or populations of select recreational and commercial species, charisma tic species such as salmon, or changes in water quality that affect fish, but none have sp ecifically valued changes in forage fish populations. For example, Olse n et al. (1991) conducted a su rvey of Pacific Northwest residents, including both a nglers and non-anglers, to de termine their WTP for doubling

the size of the Columbia River Basin salmon and steelhead runs. EPA’s proposed survey approach differs from this study and others like it (such as Cameron and Huppert, 1989) in that it would include respondents from vari ous geographic regions in the United States and would provide values for the full range of forage, recreational, and commercial species affected by 316(b) regul ations, instead of valuing a few recreational species in one specific geographical area.

Among available studies, the most closely related is Johnston et al. (2009), which estimates total willingness to pay (WTP) fo r multi-attribute aquatic ecosystem changes related to improvements in small migratory (diadromous) fish. Unlike other studies, the choice experiment data of Johnston et al. ( 2009) allow estimation of WTP associated with quantified changes in forage fish (e.g., WTP per fish or percentage change in fish), holding other ecological effects constant. That is, unlike results pr ovided by other studies in the literature, WTP estimates of Johnston et al. (2009) are not confounded with values for other changes including water quality, habi tat, overall ecologica l condition, etc. In addition, the choice experiment of Johnston et al. (2009) addresses species such as alewife and blueback herring that are neither subject to recreational or commercial harvest in Rhode Island, nor are charismatic sp ecies. Hence, the species affected are a close analog to the forage fish that are at primary issue in the 316(b) policy context.

Although the methods and data of Johnston et al. (2009) allow quantification of

combined use and nonuse values associated w ith specific improvements in forage and/or recreational fish, and also pr ovide limited ability to isol ate nonuse values from use values, the policy context and scale prevent dire ct use for analysis of national benefits of the 316(b) regulation. Specifically, Johnst on et al. (2009) estimate Rhode Island5 residents’ preferences for the restoration of migratory fish passage over dams in the

Pawtuxet and Wood-Pawcatuck watersheds. He nce, the case study is for a watershedlevel policy with statewide we lfare implications. In contra st, 316(b) policies would have nationwide implications, both on ecosystem s and on affected facilities.

(5) Consultations

The Principal Investigator for the stated-prefe rence portion of this effort is Dr. Robert

Johnston. Dr. Johnston is assi sted by Dr. Elena Besedin, a Senior Economist at Abt Associates Inc. Dr. Erik Helm at the U.S. Environmental Protection Agency serves as the project manager and a contribu tor to this research.

Robert J. Johnston is Director of the George Perkin s Marsh Institute and Professor of Economics at Clark University. He is Presid ent-elect of the Northeastern Agricultural and Resource Economics Association (NAR EA), on the Program Committee for the Charles Darwin Foundation, the Science Advisory Board for the Communication Partnership for Science and the Sea (COMPASS), and is the Vice President of the Marine

Resource Economics Foundation. Professor Jo hnston has published extensively on the valuation of non-market commodities (goods, services, and resour ces), benefit cost analysis, and resource management. His recent research emphasizes coordination of ecological and economic models to estimate eco system service values, with particular emphasis on the role of aquatic ecological indicators. He has also worked extensively in methodologies for benefit transfer, including the use of meta-analysis. Professor Johnston’s empirical work on non-market valuat ion and benefit transf er has contributed to numerous benefit cost analyses conducte d by federal, state and local government agencies in the US, Canada and elsewhere.

Elena Y. Besedin , a senior economist at Abt Asspciates Inc., specializes in the economic analysis of environmental policy and re gulatory programs, Her work for EPA has concentrated on analyzing economic benefits from reducing risks to the environment and human health and assessing environmental im pacts of regulatory pr ograms for many EPA program offices. She worked extensively on va luation of non-market benefits associated with environmental improvements of aquatic resources. Dr. Besedin’s empirical work on non-market valuation included design and implementation of stated and revealed preference studies and benefit transfer methodologies. Her recent work focused on developing integrated frameworks to value ch anges in ecosystem services stemming from environmental regulations.

EPA has conducted an external peer review of the previous version of the survey instrument. The previous version of the surv ey instrument was reviewed by Dr. Richard C. Bishop, Dr. Kevin J. Boyle, Dr. Timothy C. Haab, Dr. James R. Ka hn, and Dr. Alan J. Krupnick. The review focused on (1) the a ppropriateness of the survey design for measuring fish resources affected by coo ling water intake structures (CWIS), (2) sufficiency and appropriateness of informa tion provided in the su rvey on the baseline ecological conditions, the expected changes, a nd the expected costs, and (3)whether the answers to this survey lend themselves to straightforward econo mic interpretation of6 WTP. The reviewers acknowledged the extensiv e effort that was placed into survey design and noted that the 316(b) survey wa s a well-designed study. They also offered concrete suggestions for further improveme nts of the survey instrument. EPA has incorporated many of the reviewers comments in subsequent survey revisions (Versar 2006).

(6) Peer Review Plans

Internal Agency reviewers provided and will co ntinue to provide input on the content and format of the focus groups, questionnaire de sign issues, and issues related to survey sampling design and methodology. Survey material s will be subject to routine internal review by Dr. Helm and EPA’s National Cent er for Environmental Economics (NCEE)

staff.

We note that the current survey instrument is built upon an earlier ve rsion that was peer reviewed in January 2006. It incorporates recommendations received from the first peer review panel. Because the final product of this study meets the major technical work criteria specified in the P eer Review Handbook (U.S. EPA 2000) the Agency also plans to convene a peer-review panel to review th e entire survey process. First, the Agency plans to review the survey in strument prior to fielding it. After the survey is completed, EPA will peer-review study results, and EPA’ s final estimated results for the 316(b) Existing Facilities rulemaking,

(7) Confidentiality

The survey instrument will fully conform to federal regulations – specifically the Privacy Act of 1974 (5 U.S.C. 552a), the Hawkins- Stafford Amendments of 1988 (P.L 100-297), and the Computer Security Act of 1987. Each prospective respondent will be informed that their participation in th e survey is voluntary, and that their identities will be kept confidential by the investigator s and not associated with th eir responses. Neither EPA nor any other federal or state agency will ha ve access to the names of respondents.

The focus groups will be recorded using audiotapes, which will later be transcribed. However, individuals will not be identified on th e transcripts. To insure that the focus groups include a representative and divers e sample of individuals, EPA will tabulate basic demographic information for participan ts, such as age, ethnicity, occupation, and income. However, no personal identifying inform ation about participants, such as names, phone numbers, or addresses, will be include d in the focus group transcripts. Prior to commencing any audio recordings , respondents would be informed of EPA's intentions to audiotape the focus group proceedings, and informed that all taped responses will be strictly anonymous and confidential.

(8) Sensitive Questions7 There are no questions on sexual behavior a nd attitudes, religious beliefs, and other matters that are commonly considered private or sensitive in these focus group materials. Answers to standard demographic questions (i.e., age, income) will be included.

(9) Respondents

EPA intends to conduct eight focus groups as part of the testing of the survey instrument. There are a total of seven study regions for pur poses of evaluating the 316(b benefits. For the purpose of the stated pr eference survey implementation, EPA uses five geographic regions: California, Great Lakes, Inland, North east, and Southeast. The Northeast region

includes the North Atlantic and Mid Atlantic regions and the Southeast region includes the South Atlantic and Gulf of Mexico regi ons. Because focus groups in different study regions might provide distinct information re levant to survey design the Agency will conduct focus groups in multiple regions. The Ag ency will modify the survey instrument for each region relative to the current survey draft for the Northeast region as follows:

• Cover – The text on the cove r currently reads “A survey of Northeast Residents (CT, DE, MA, MD, ME, NH, NJ, NY, RI, VT)”. “Northeast Residents” and the

list of included states will be cha nged to match the respondent’s region.

• Cover – The photo on the cover is of a fora ge species (silversides) in found in the Northeast region. If that species isn't f ound in other regions, we will replace the cover photo with a similar substitute photo for a forage species relevant to that region.

• Page 1 – The survey states that it “asks for your opinions regarding policies that

would affect fish and habitat in the No rtheast US.” “Northeast US” will be replaced with the respondent’s region.

• Page 1 – Includes the statement that “N ortheast fresh and salt waters support

billions of fish.” “Northeast” will be re placed with the respondent’s region and

“salt water” will be removed from the Inland and Great Lakes regions.

• Page 2 - The survey states that “Cooling water use affects fresh and salt waters throughout the Northeast US, but almost all fish losses are in salt waters such as coastal bays.” For the California and S outheast regions, which include both salt water and freshwater facilities, refere nce to the Northeast region would be replaced with the name of the respo ndent’s region. For the Great Lakes and Inland survey regions, which only include fr eshwater facilities, reference to salt

water will be removed.

• Page 2 – A map of the Northeast region a nd facility locations is presented. A comparable map will be produced for each region, and map included in the survey will correspond to the respondent’s region.

• Page 4 - This page provides an exampl e for two Northeast coastal species and includes a figure comparing estimates of young fish lost to the total number of

adult fish. The estimate of total losses and example species will be modified to describe species and losses occurr ing within the respondent’s region.

• Page 5 - The total losses (1.1 billion) currently included within the survey corresponds to losses from facilities in Nort heast coastal waters. This total would be replaced with the total losses for the respondent’s region. The pie charts will8 be updated based on the regional total while maintaining the 25% and 95% reductions used in the example.

• Page 7 - The table text describing each score will be changed to include the current scores for the respondent’s region.

• Page 8 - Text describing the commercial fish sustainability score and fish population score will be change d to describe the current score for the respondent’s region relative to its maximum.

• Page 9 – The text states that the aquatic ecosystem score measures “how close

affected Northeast waters are to the mo st natural, undisturbed condition that is possible.” “Northeast waters” will be repl aced with a reference to waters within the respondent’s region.

• Page 12 – The included figure illustrate s the location of “Coastal Facilities Affected by the Proposed Policies”. For other regions which include both salt water and freshwater facilities, the figure will be replaced with a similar figure circling salt water facilities within the respondent’s region. For the Great Lakes and Inland regions (which do not include salt water facilities), only facilities within the region will be shown in th e figure, making the circle unnecessary.

• Pages 13-16 – Regional references within the table headings in Questions 4, 5, and 6 (e.g., “Policy Effect NE Coastal”, “O ption A NE Coastal”) will be modified to refer to the policy eff ects and options for the res pondent’s region. The “Fish Saved per Year” score includes a note remi nding the respondent of total fish lost within the region due to I&E (e.g., 1.5 b illion); this number will be changed to

reflect total losses within the respondent’s region. The values describing the current situation and Options A and B w ithin experiment questions 4, 5, and 6 will also vary across regions.

• Page 17 – Text describing “National versus Northeast policies” will be modified to refer to the respondent’s region.

• Page 21 – Question 12 asks the respondent how times he partic ipated each of six activity categories over the last year within the “Northeastern US only”. Text will be changed to reflect the respondent’s regi on including the list of states which the region includes.

EPA will edit the Northeast survey instrument so that respondents in each of the 5 study regions receives a survey that is representativ e of their region. The specific details of the presentation of this region specific material is to be informed and tested by the focus group process. In general the instrument will be edited so that all references to geographic location, fish species and environmental baseline information, and regionally specific policy outcomes will match the study region.

Each focus group will last approximately 90 minutes. We anticipate that eight to ten people will attend each focus group. For each focus group, individuals will be recruited from one of the study regions to reflect the range of ultimate potential respondents to a survey. The groups will be led by an experienced moderator, and held in facilities with

audio/video recordi ng capabilities.9 Focus group participants will be randomly selected by marketing research firms from panels of focus group participants maintained by each firm. Participants are eligible if they are 18 years of age or older; if they are not full-time students; if they do not have occupations related to the environment, includi ng water treatment and sewage, electric or nuclear power companies; and if they have not participated in a survey or focus group in the last six months. Each of the focus gr oups will include a mix of individuals with diverse socioeconomic backgrounds, based on ch aracteristics such as income, marital status, age, race, education, occupation, and gende r. Participants will be asked to attend a focus group session and participate in a discussion of specific topics led by a moderator. Participation in the focus group sessions is vol untary. Participants will have to expend time, effort, and travel to participate in the focus group sessions. Following standard practice in marketing research, participants w ill be compensated for their time and effort. As described by Dillman (2000) in the cont ext of population surveys, compensation can increase response rates and provide an in centive to participat e among respondents who may not have a strong ex ante interest in th e subject matter. Therefore, compensation can help to avoid self selection biases associat ed with ex ante interests in environmental policies, and can thereby encourage a more representative sample of focus group participants. The proposed incentiv e fee is $75.00 per participant.

(10) Collection Schedule

The proposed timeline for the data collection is as follows.

Task: Date: Submit ICR package for focus groups to OMB March 2010 Conduct focus groups June - July 2010 Finalize draft survey instrument June 2010 Submit ICR package to OMB July 2010 Receive approval to conduct survey December 2010 Field survey to sample March2011

Allow 8-10 weeks for data collection Analyze data May - June 2011 Prepare final report a nd papers July 2011 Internal EPA peer review of survey methodology

and results TBD

(11) Respondent Burden

For each of the eight groups we will recruit te n participants. Participants will be given directions to a marketing research firm wh ere the focus group will occur. They will be provided a light snack and beve rages as well as $75 compen sation for their time. We anticipate that there will be some attrition be tween recruiting and the actual time of the focus group, so we anticipate eight to nine participants in each session.

For the purposes of calculating burden, the maximum total number of respondents is 80

. Each individual will participate in a 90 minute session. We also anticipate that10 individuals will spend up to 30 minutes of tr avel time. Therefore, the total number of respondent hours requested is 160 hours

.

(12) Information Requested

Individuals who attend the focus groups will at some point during the session be asked to review the draft survey, answer draft surv ey questions, and provide feedback on these presentation materials and survey questions. After reviewing intr oductory materials and administering draft survey questions, the mode rator of the focus group will read and ask participants debriefing questions . A script containing relevant questions that will be used to guide the discussion is attached to this request. EPA notes that the exact wording of debriefing questions in focus groups ma y be adjusted somewhat by focus group

moderators to conform to the flow of ongoing discussion. Survey materials will be revised between groups based on respondent re actions and comments to ensure that the draft instrument used in each focus group is as clear, concise, and comprehensive as possible.

EPA notes that not all discussion topics or que stions presented in the attached script can be covered in any single focus group session, due to time constraints. Instead, these questions will be covered over a number of di fferent focus groups. In some focus groups,

the moderator will first offer some draft valu ation questions to focus group participants, as noted above, followed by debriefing ques tions. In other cases, the focus group may begin with general discussion questions, prio r to the considerati on of draft survey questions. The order of questions either with in or between the categories above may also be altered, as appropriate to the flow of discussion within a particular focus group. Freedom to alter question order is important , as it is often impossible to predict the direction in which responde nts will direct their disc ussions and/or answers.

Some of the focus groups will incorporate indi vidual one-on-one protocol interviews of the survey instrument. One-on-one protocol interviews will allow in-dep th exploration of the cognitive processes used by respondents to answer survey questions, without the potential for interpersonal dynamics to sw ay respondents’ comments (Kaplowicz et al. 2004).

REFERENCES

Adamowicz, W., P. Boxall, M. Williams, and J. Louviere. 1998. Stated Preference Approaches for Measuring Passive Use Values: Choice Experiments and Contingent Valuation. American Journal of Agricultural Economics 80(1): 64-75. Cameron, T.A. and Huppert, D.D. (1989) OLS versus ML Estimation of Non-market Resource Values with Payment Card Interval Data. Journal of Environmental Economics and Management . 17, 230-246.

Carson, R. T., T. Groves, and M. J. Machina. 2000. “ Incentive and Informational Properties of
Preference Questions .” Working Paper, Department of Economics, University of California, San Diego.11 Carson, R.T. and T. Groves. 2007. Incentive a nd Informational Properties of Preference
Questions. Environmental and Resource Economics 37(1):181–210.

Collins, J.P. and C.A. Vossler. 2009. Incentive Compatibility Tests of Choice Experiment Value
Elicitation Questions. Journal of Environmental Economics and Management 58(2): 226-235.

Cummings, Ronald G., and Harrison, Glenn W., "The Measurement and Decomposition of Nonuse Values: A Critical Review", Environmental and Resource Economics , 5, 1995, 225-

247. Freeman III, A.M. 2003. The Measurement of E nvironmental and Resource Values: Theory and Methods. Resources for the Future, Washington, DC. Hanley N., Colombo, S., Tinch, D., Black, A. a nd Aftab, A. (2006) Estimating the benefits of water quality improvements under the Water Framework Directive: are benefits

transferable? European Review of Agricultural Economics 33: 391-413. Hanley, N., Wright, R.E. and Alvarez-Farizo, B. (2006) Estimating the economic value of improvements in river ecology using choice e xperiments: an application to the water

framework directive. Journal of Environmental Management 78: 183-193.

Herriges, J., C. Kling, C. Liu and J. Tobias. 2010. What are the Consequences of Consequentiality? Journal of Environmental Economics and Management 59(1): 67-81.

Johnston, R.J. 2006. Is Hypothetical Bias Univ ersal? Validating Contingent Valuation Responses Using a Binding Public Referendum. Journal of Environmental Economics and Management
52(1): 469-481.

Johnston, R.J., E.Y. Besedin and R.F. Wardwe ll. 2003. Modeling Relations hips Between Use and Nonuse Values for Surface Water Quality: A Meta-Analysis. Water Resources

Research 39(12): 1363-1372. Johnston, R.J., E.Y. Besedin, R. Iovanna, C.J. Miller, R.F. Wardwell and M.H. Ranson. 2005. “Systematic Variation in Willingness to Pa y for Aquatic Resource Improvements and Implications for Benefit Transfer: A Meta-Analysis.” Canadian Journal of Agricultural

Economics 53(2-3):221-248. Johnston, R.J., E.T. Schultz, K. Segerson a nd E.Y. Besedin. (2009) Bioindicator-Based Stated Preference Valuation for Aqua tic Habitat and Ecosystem Serv ice Restoration, in Bennett,

J. ed. International Handbook on Non-Ma rketed Environmental Valuation. Cheltenham, UK: Edward Elgar, forthcoming. Just, R.E., D.L. Hueth and A. Schmitz. 2004. The Welfare Economics of Public Policy: A Practical Approach to Public Policy . Cheltenham, UK: Edward Elgar. Kaplowitz, M.D., F. Lupi and J.P. Hoehn. 2004. “Multiple Methods for Developing and Evaluating a Stated-Choice Questionnaire to Value Wetlands.” Chapter 24 in Methods for Testing and Evaluating Survey Questionnaires , eds. S. Presser, J.M. Rothget, M.P. Coupter, J.T. Lesser, E. Martin, J. Martin, and E. Singer. New York: John Wiley and Sons. Morrison, M. and Bennett, J. (2004) Valuing New So uth Wales rivers for use in benefit transfer. The Australian Journal of Agricultural and Resource Economics 48: 591-611. Olsen, D., J. Richards, and R.D. Scott. 1991. “Existence and Sport Values for Doubling the Size

of Columbia River Basin Salmon and Steelhead Runs.” Rivers . 2(1):44-56. U.S. Environmental Protection Agency (U.S. EPA). 2000. Science Policy Council Handbook:

Peer Review. U.S. EPA,Office of Research and Development, Washington D.C., EPA

100-B-00-001. Versar. 2006. Comments Summary Report: Peer Re view Package for "Willingness to Pay Survey Instrument for §316(b) Phase III Cooling Wate r Intake Structures.” Prepared by Versar Inc., Springfield, VA.12 Vossler, C.A. and M.F. Evans. 2009. Bridging the Gap Between the Field and the Lab: Environmental Goods, Policy Make r Input and Consequentiality. Journal of Environmental
Economics and Management 58(3): 338-345.