PART B OF THE SUPPORTING STATEMENT

1. Survey Objectives, Key Variables, and Other Preliminaries 1(a) Survey Objectives The overall goal of this survey is to explore how public values (including non

- use values) for fish and aquatic organisms are affected b y I& E mortality at cooling water intake structures (CWI S) located at existing 316(b) facilities, as reflected in individuals’ willingness to pay for programs that would prevent such losses. EPA has designed the survey to provide data to support the following specific objectives:

 To estimate the total values, including non

- use values, that individuals place on preventing l osses of fish and other aquatic organisms caused b y CW IS at existing 316(b) facilities.

 To understand how much individuals value preventing fish losses, increasing fish populations, improvements in aquatic ecos ys tems, and increasing commercial and recreati onal catch rates.

 To understand how such values depend on the current baseline level of fish populations and fish losses, the scope of the change in those measures, and the certainty level of the predictions.

 To understand how such values vary with respect to individuals’ economic and demographic characteristics.
Understanding total public values for fish resources lost to I &E mortality is necessary

to determine the full range of benefits associated with reductions in impingement and entrainment losses at existing 316(b) facilities. Because non

- use values may be substantial, failure to recognize such values may lead to improper inferences regarding policy benefits (Freeman 2003).1(b) Key Variables

The key questions in the survey

ask respondents whether o r not they would vote for policies that would increase their cost of living, in exchange for specified changes in: (a) I&E mortality losses of fish, (b) commercial fish sustainability , (c) long

- term fish populations, and (d) condition of aquatic ecosy stems 1 . More specifically, the choice experiment framework allows respondents to view pairs of multi

- attribute policies associated with the reduction of I&E mortality losses. Respondents are asked to choose the program that they would prefer, or to choose to reject both policies. This follows well

- established choice experiment methodology and format (Adamowicz et al. 1998; L ouviere et al. 2000; Bennett and Blamey 2001; Bateman et al. 2002). Important variables in the anal ys is of the choice questions are how the respondent votes, the amount of the cost of living increase, the number of fish losses that are prevented, the sustainability

of commercial fishing, the change in fish populations, and the condition of aquatic
ecos ys tems. Other important variables inc lude whether or not the respondent is a user of the affected aquatic resources, household income, and other respondent demographics.
1(c) Statistical Approach
EPA believes that a statistical survey approach is appropriate. A census approach is impractical because contacting all households in the U.S. would require an enormous expense. On the other hand, an anecdotal approach is not sufficientl y rigorous to pro vide a useful estimate of the total value of fish loss reductions for the 316(b) case. Thus, a statistical survey is the most reasonable approach to satisfy EPA’s anal yt ic needs for the 316(b) regulation benefit anal ys is.

1 The environmental attributes to be compared against the cost of living increases where designed based on the Johnston et al. (2009) Bioindicator

- Based Stated Preference Valuation (BSPV) method which wa s developed to promote ecological clarity and closer integration of ecological and economic information within SP studies. In contrast to traditional SP valuation, BSPV employs a more structured and formal use of ecological indicators to characterize and communicate welf are - relevant changes. It begins with a formal basis in ecological science, and extends to relationships between attributes in respondents’ preference functions and those used to characterize policy outcomes. Specific BSPV guidelines ensure that survey scen arios and resulting welfare estimates are characterized by: (1) a formal basis in established and measurable ecological indicators, (2) a clear structure linking these indicators to attributes influencing individuals’ well

- being, (3) consistent and meaning ful interpretation of ecological information, and (4) a consequent ability to link welfare measures to measurable and unambiguous policy outcomes. The welfare measures provided by BSPV method can be unambiguously linked to models and indicators of ecosyste m function, are based on measurable ecological outcomes, and are more easily incorporated into benefit cost analysis. This methodology was developed in part to address the EPA Science Advisory Board’s call for im proved quantitative linkages between ecolog ical services and economic valuation of those services.EPA has retained Abt Associates Inc. (55 Wheeler Street, Cambridge, MA 02138) as a

contractor to assist in questionnaire design, sampling design, and anal ys is of the survey results.
1(d) Feasibility
The survey

instrument was repeatedl y pre - tested during a series of seven focus groups (conducted under a different I CR with OMB control # 2090

- 0028), in addition to the twelve focus groups conducted for the Phase III survey (EPA

- HQ

- OW

- 2004

- 0020), and it will be subject to peer review b y reviewers in academia and government, so EPA does not anticipate that respondents will have difficulty interpreting or responding to any of the survey

questions. Additionally , since the survey will be administered as a mail survey , it will be easil y accessible to respondents. Thus, EPA believes that respond ents will not face an y obstacles in completing the survey , and that the survey will produce useful results. EPA has dedicated sufficient funding (under EPA contracts No. EP

- C

- 07

- 23) to design and implement the survey

. Given the timetable outlined in Sect ion A.5(d) of this document, the survey results will be available for timel y use in the final benefits anal ys is for the 316(b) existing facilities rule.

2. Survey Design 2(a) Target Population and Coverage
The target population for this survey includes i ndividuals from continental U.S. households who are 18 years of age or older. The sample will be chosen to reflect the demographic characteristics of the general U.S. population.
2(b) Sampling Design (I) Sampling Frames
The sampling frame for this surve y is the panel of individuals recruited b y random digit dialing ( over - lapping dual frame; land and cell) to participate in a mail survey . The overall sampling frame from which these individuals would be selected is the set of all individuals in continenta l U.S. households who are 18 ye ars of age or older and who have listed phonenumbers. Individuals in the panel are recruited using a list

- assisted random digit dialing

telephone methodology , thus providing a probability

- based starting sample of U.S. telep hone households. The telephone screening will use an overlapping dual frame design of landline and cell phone numbers to increase the coverage of the sample. The inclusion of cell phone numbers will allow EPA to reach potential respondents who only have cell phones and would be missed if only a sample of landlines was selected. The dual frame design also avoids bias because the cell

phone

- onl y population skews heavil y towards lower

- income and younger individuals. The cell and landline samples will be we ighted to correct for each individual’s probability of being included in the survey . People who live in a household with more than one person are less likely to be chosen for the survey , and people with more than one telephone line (e.g., both cell and landline) are more likel y to be chosen for the survey . All landline respondents to the telephone screener will be asked the number of eligible respondents in their household and every respondent (cell and landline) will be asked whether they have a landline only , a cell phone onl y, or both. Next, the survey data will be weighted to match national data (U.S. Census and National Health Interview Survey) b y a set of demographic variables (e.g., age and gender) as well as phone usage (cell

- only users, landline

- only

users and people who use both cell phones and landline phones). For January through June 2009, the National Health Interview Survey indicated that 15.5 percent of households are landline only , 22.7 percent are cell phone onl y, and 59.4 percent have bo th landline and cell phone service. About two percent of households have no telephone service. It is anticipated that estimates for landline and cell phone service from the January to June 2010 National Health Interview Survey will be available for weighting purposes by the time the survey

would be implemented. For discussion of techniques that EPA will use to minimize non

- response and other non

sampling errors in the survey sample, refer to Section 2(b)(II), below.
(II) Sample Sizes
The intended sample size for the survey is 2,288 households including onl y households providing completed mail survey s. This sample size was chosen to provide statistically robust regression results while minimizing the cost and burden of the survey . Giv en this sample size,the level of precision (see section 2(c)) achieved by the anal ys is will be more than adequate to meet the anal yt ic needs of the benefits anal ys is for the 316(b) regulation. For further discussion of the level of precision required b y this anal ys is, see Section 2(c)(I) below.

(III) Stratification Variables
The survey

sample will be selected using a stratified selection process. For the selection of households, the population of households in the contiguous 48 states and the District of Columbia will be stratified by the geographic boundaries of four study regions: Northeast, Southeast, Inland, and Pacific. As described previously , the Northeast region includes the North Atlantic and Mid Atlantic 316(b) benefits regions, the Southeas t region includes the South Atlantic and Gulf of Mexico 316(b) benefits regions , the Pacific region includes states on the Pacific coast, and the Inland region includes all non

- coastal states. The sample is allocated to each region in proportion to the tot al number of households in that region, with at least 288 completed survey s in each region. This is the number required to estimate the main effects and interactions under an experimental design model as described in Section 4(a) of Part A. To accommodate this requirement the sample sizes in other regions will be slightly reduced. A sample of 288 households completing the national survey version would be distributed among the study regions based on the percentage of regional survey sample (as shown in Tabl e A1) to ensure that respondents to the national survey version are distributed across the continental U.S.
(IV) Sampling Method
Using the stratification design discussed above, respondents will be randomly selected. First, a larger sample of household s will be selected through random digit dialing in each of the 4 regions. The household to which the selected telephone number belongs will be called and the respondent will be asked whether he or she is willing to provide information required b y the
surv ey . If the person in the household agrees, then a questionnaire will be mai l ed to that household. If it is assumed that 40 % of the households which agreed to provide informationactuall y return a completed mail survey (completion rate) then 5,720 questi onnaires will need to

be mailed to households. 2 If we assume that 60% of the households that are called say “yes” they

are willing to receive a questionnaire ( the screening response rate), then we need to screen 9,533 households. A dual frame random digit dialing design , including both landline and cell phone numbers, will be used for telephone screening

. If we assume that 40 % of the households called will participate in the telephone screener, then we will need to call 23,833 numbers. The actual number of telephone numbers selected through random digit dialing sampling will be larger than the number of households called. This is because we are expecting that 40% of the telephone numbers initially

selected will be screened out as non

- working or business numbers. Therefore the actual number of telephone numbers selected will be around 39,722. The quantity of telephone numbers that we need to select in each region can be obtained by allocating the total number in proportion to the total households in that region. For obtaining population

- based estimates of various parameters, each responding household will be assigned a sampling weight. This weight combines a base sampling weight which is the inverse of the probability of selection of the household and th en an adjustment for
non - response. The weights will be used to produce estimates that are generalizable to the population from which the sample was selected (e.g., percent of population participating in water
- based recreation such as fishing and shellfish ing). Proportional allocation of the sample to regions ensures an equal probability sample. T o estimate total WTP for the quantified environmental benefits of the 316(b) existing facilities rulemaking data will be anal yz ed statistically using a standard r andom utility model framework.
(V ) Multi

- Stage Sampling

Multi

- stage sampling will not be necessary for this survey

.

2 The assumed response rate is based on a similar stated preference studies conducted by Johnston et al. (2010) in the Northeast U.S. Actual response rates could vary across study regions.2(c) Precision Requirements

(I) Precision Targets
Table B1, below, shows the target samples sizes for both the U.S. (excluding Alaska and Hawaii) and each of the four EPA study regions. At the regional level, a sample of 2,000 households (completed survey s) will provide estimates of population percentages with a margin of error ranging from 3.6 to 5.8 percentage points at the 95% confi dence level. A sample of 288 household for the national survey version (completed survey s) will provide an estimate of population percentages with a margin of error no greater than 5.8 percentage points at the 95% confidence level.

Table B1: Number of H ouseholds and Household Sample for Each EPA Study Region

Region Household Population Household Sample Northeast 23,281,296 417
Southeast 31,378,122 562
Inland 40,852,983 732
Pacific 16,158,206 288 Total for Regional Survey Versions 111,670,607 2,000 National Survey Version 111,670,607 288 Source: The number of households in each region was obtained based on the estimated population size and average household size from the 200 6

- 200 8 American Community Survey (ACS)

. (II) Non
-

Sampling Errors
One issue that may

be encountered in stated preference survey s is the problem of protest responses. Protest responses are responses from individuals who reject the survey format or question design, even though they may value the resources being considered (Mitchell and Carson 1989). For example, some respondents may feel that any

amount of I&E is unacceptable, and choose not to respond to the survey . To deal with this issue, EPA has included several questions, including an open

- ended comments section, to help identify protest responses. Theuse of such methods to identify protest responses is well

- established in the literature (Bateman et al. 2002). Moreover, many researchers (e.g., Bateman et al. 2002) suggest that a choice experiment format, such as that proposed here, may ameliorate such responses (over the earlier contingent valuation format). A different ty

pe of non

- sampling error is non

- response bias. Non

- response rates in this survey are affected by

non - response at each stage of sampling. There are two ty pes of non

respondents to this survey : respondents unwilling to complete a questionnaire when initially contacted b y phone and respondents who agree but do not complete the questionnaire. EPA will conduct several additional activities to minimiz e the potential for non

- response bias in the current survey : (1) The telephone interviews will be conducted b y trained interviewers who are experienced in encouraging response. Calls will be placed on different days of the week and at different times to maxi mize response. (2) The mail survey

will be designed to maximize response. A mail survey will be sent to all those who agree to participate. Then, one week later, a postcard reminder will be sent. Three weeks later, a second cop y of the questionnaire with a revised cover letter will be sent to non

- respondents. As final follow

- up, another postcard reminder will be sent one week later. (3) Response rates will be tracked on a dail y basis for both the telephone screener and mail survey . If any

unexpected declines are encountered, corrective action can immediately

be undertaken. (4) EPA will undertake non

- response bias anal ys is as detailed in the following section. If necessary , EPA will use appropriate weighting or other statistical adjustment to correct the bias beca use of non

- response.

Non

- response Interviews To determine whether there is an y evidence of significant non

- response bias in the completed sample, EPA will compare the characteristics of the completed and non

- completed cases from the random digit dialing s ample. There are two main stages of non

- response: non response to the telephone survey and non

- response to the mail survey

.To assess the potential for telephone non

- response bias, EPA will collect sample

characteristics in the course of the telephone surv ey as follows: (1) prior to household contact, (2) at respondent selection for the survey , and (3) after beginning the interview. EPA will then anal yz e the data available for respondents and non

- respondents prior to household contact and at respondent sel ection. Because the telephone instrument is very short, we do not expect many people prematurel y terminating the call after the selected household member has started the interview. The drawn sample of random digit dialing numbers from the landline frame th at are released to telephone interviewers for contact attempts can be coded for certain contextual characteristics. The “hundreds bank” from which the RDD number is drawn (e.g., 301

- 608

- 38xx) is classified geographicall y according to the unit from which the majority of listed numbers in that bank are located. This permits us to classify all of the numbers drawn by EPA region, State, and urbanicity (central city , metro remainder, and non

- metropolitan area). It also permits us to classify

those numbers by demographic characteristics, such as percentage of African

- American and Hispanic, based on Census data linked to telephone exchanges. EPA will compare the characteristics of all dialed residential numbers in which an interview had not been completed with those that had been completed using these variables to determine whether non

response bias is present. Once contact is achieved with a household, one adult is selected in each household as the designated respondent. If there is more than one eligible respondent per household, then a random selection is done for the individual with the most recent/next birthday . EPA will identify the age and gender of this individual if callbacks are necessary . This information permits additional comparisons of an y differences in the age and gender of respondents in the completed sample and those in the non

- completed sampl e after household contact and respondent selection have been made. The telephone screener includes not only an invitation to participate in the mail survey , but also some attitudinal (relative importance of various government activities) differences and b ehavioral differences (recreational activities), as well as demographic characteristics (see Attachment 6 for detail). EPA will use the telephone screening data to compare mail survey respondents and non

- respondents. The items of information collected duri ng telephone screening will help determine the ty pe of person that is likely to not respond to the survey and may help informing weighting classes for adjusting weights of respondents to account for non

- response and

minimize the bias because of non

- respon se.
2(d) Questionnaire Design
The information requested by the survey

is discussed in Section 4(b)(I) of Part A of the supporting statement. The full text of the draft questionnaire for the Northeast region is provided in Attachment 1 and the full text of the draft questionnaire for the national survey version is provided in Attachment 2. The following bullets discuss EPA’s reasons for including the questions in the survey :

 Relative Importance of Issues Associated with Industrial Cooling Water . EPA i ncluded this section to prepare respondents to answer the stated preference questions by motivating respondents to consider the relative importance of key issues associated with the use of cooling water by industrial facilities.

 Concern for Policy Issues. EPA included this section to prepare respondents to answer the stated preference questions by motivating respondents to think about the relative importance of different policy issues.

 R elative Importance of Effects. This section was included to promote understanding of the metrics included in the stated preference questions by asking them to consider their relative importance prior to evaluating policy options and by

encouraging respondents to
re - read previous pages for reminders if necessary

.
 Voting for Regulations to Prevent Fish Losses in the Respondent’s Region (or Nationally). The questions in this section are the key part of the survey . Respondents’ choices when presented with specific fish

- related resource changes within their region and household cost increases are the main data that allow estimation of willingness

- to
pay . The questions are presented in a choice experiment (A, B, or neither) format because this is an elicitation format that has been succes sfull y used b y a number of previous valuation studies (Adamowicz et al. 1998; Bateman et al. 2002; Bennett and Blamey 2001; Louviere et al. 2000; Johnston et al. 2002a, 2005; Opaluch et al. 1993). Furthermore, man y focus group participants indicated that they have some previousexperience making choices within a framework in which they are asked to vote for one of a series of options, and are comfortable with this format.

 Reasons for Voting “No Policy”. This question provides information that will be used by EPA to identify

protest responses.

 Respondent Certainty and Reasons for Voting . This section is designed to identify respondents who incorrectly interpreted the choice questions or the uncertainty of outcomes. Responses to these questions are important to successfull y control for hy pothetical bias.

 Recreational Experience. This question elicits recreational experience data to test if certain respondent characteristics influence responses to the referendum questions. This question will also allow EP A to identify resource non - users, for purposes of estimating
non - user WTP (to gauge the relative importance of non

- use values to overall benefits).

 Demographics. Responses to these questions will be used to estimate the influence of demographic variables on respondents’ voting choices, and ultimately , their WTP to prevent I&E mortality losses of fish. This information will allow EPA to use regression results to estimate WTP for populations in different regions affected b y the 316(b) rule for existing faci lities.

 Comments. This section is primarily intended to help identify protest responses, i.e. responses from individuals who rejected the format of the survey or the way the questions were phrased.

3. Pretests and Pilot Tests
EPA conducted extensive pre tests of the survey instrument during a set of seven focus groups (EPA I CR # 2090

- 0028 ), in addition to the twelve focus groups conducted for the Phase III survey . These focus groups included individual cognitive interviews with survey respondents (Kaplow icz et al. 2004), and think

- aloud or verbal protocol anal ys es (Schkade and Pay ne 1994). Individuals in these focus groups completed draft survey questionnaires and provided comments and feedback about the survey format and content, their interpretations o f the questions, and other issues relevant to stated preference estimation. Particular emphasis in these survey pretests was on testing for the presence of potential biases associated with poorl y - designed statedpreference survey s, including hy pothetical bias, strategic bias, s ymbolic (warm glow) bias, framing effects, embedding biases, methodological misspecification, and protest responses (Mitchell and Carson 1989). Based on focus group and cognitive interview responses, EPA made various improvements to the survey questionnaire including changes to ameliorate and minimize these biases in the final survey instrument.

4. Collection Methods and Follow
- up 4(a) Collection Methods
The survey

will be administered as a mail survey . Respondents identified by random digit dialing who agree to participate will be mailed a cop y of the survey questionnaire. Respondents will be asked to mail the completed survey back to EPA.4(b) Surve y Response and Follow
- up

The target response rate for the mail survey is 40 percent. That is, 40 percent of households which agreed to participate in the mail survey are expected to return a completed survey . However, the actual response rate compared t o the general population is lower, given that the telephone screening process is expected to have a response rate of 60 percent among contacted households. The resolution rate (determining the status of a telephone number), the screener response rate and the survey completion rate will contribute to the overall response rate in the survey . The actual number of telephone numbers selected for inclusion will be much larger to account for the exclusion of non

- working and business numbers. The number of telep hone numbers needed for selection in each region can be obtained b y allocating the total number in proportion to the number of households in that region. To improve the response rate, a ll of these respondents will receive a reminder postcard approximately one week after the initial questionnaire mailing. Then, approximately three weeks after the reminder postcard, all those who have not responded will receive a second cop y of the questionnaire with a revised cover letter. The following week, another remin der postcard will be sent. As noted in Section 2(b), the survey sample will be selected using a stratified selection process. For the selection of households, the population of households in the contiguous 48 states and the District of Columbia will be stratified by the geographic boundaries of four EPA study regions. In addition, EPA will administer a national version of the survey that does not require stratification. We will keep track of the response rates for each of regional survey s and the natio nal version of the survey to ensure that the rates are reasonable. We will also look at the frame characteristics of non

- respondents to determine if there are an y substantial biases in the estimates because of an imbalance in the distribution of certain i mportant subgroups in the sample.

5. Analyzing and Reporting Survey Results 5(a) Data PreparationSince the survey will be administered as a mail survey , survey responses will manually entered into an electronic database after they are returned. After all responses have been entered, the database contents will be converted into a format suitable for use with a statistical anal ys is software package. The mail survey , database management, and data set conversion will be conducted b y Abt Associates Inc. A ll survey

responses will be vetted for completeness. Additionally , respondents’ answers to the choice experiment questions will be tested to ensure that they are internall y consistent with respect to scope and other expectations of neoclassical preference theory , such as transitivity

. Responses which satisfy

transitivity

exhibit relational relationships when separate choices among policy options are compared. For example, if values for policy 1 are greater than policy 2 , and values for policy 2 are great er than Policy 3 , then values for policy 1 should also be greater than values for Policy 3

.

5(b) Analysis
Once the survey

data has been converted into a data file, it will be anal yz ed using statistical analy sis techniques. The following section discusses the model that will be used to anal yz e the stated preference data from the survey

. Analysis of Stated Preference Data The model for anal ys is of stated preference data is grounded in the standard random utility model of Hanemann (1984) and McConnell (1990). This model is applied extensively within stated preference research, and allows well

- defined welfare measures (i.e., willingness to pay) to be derived from choice experiment models (Bennett and Blamey 2001; L ouviere et al. 2000). Within the standard random utility model applied to choice experiments, hy pothetical policy alternatives are described in terms of attributes that focus groups (Johnston et al. 1995; Adamowicz et al. 1998; Opaluch et al. 1993) reveal as relevant to respondents’ utili ty , or well

being. One of these attributes would include a mandatory monetary

cost to the respondent’s household.Applying this standard model to choices among policies to reduce I&E mortality losses, EPA defines a standard utility function U

i ( . ) that includes environmental attributes of an I&E reduction plan and the net cost of the plan to the respondent. Following standard random utility theory , utility

is assumed known to the respondent, but stochastic from the perspective of the researcher, such th at
(1) U i ( . ) = U( X i , D , Y

- F i ) = v( X i , D , Y

- F i ) + ε i
where: X i = a vector of variables describing attributes of I&E reduction plan i ; D = a vector characterizing demographic and other attributes of the respondent. Y = disposable income of the respondent. F i = mandatory

additional cost faced b y the household under plan i ; v( . ) = a function representing the empiricall y estimable component of utility ;
ε i = stochastic or unobservable component of utility , modeled as an econometric error.

Econometricall y, a model of such a preference function is obtained by methods designed for limited dependent variables, because researchers only observe the respondent’s choice among alternative policy

options, rather than observing values of U i (.) directly

(Madda la, 1983; Hanemann, 1984). Standard random utility models are based on the probability that a respondent’s utility

from a policy

Plan i , U

i (.) , exceeds the utility

from alternative Plans j , U j (.) , for all potential plans j≠ i considered b y the respondent . In this case, the respondent’s choice set of potential policies also includes maintaining the status quo. The random utility model presumes that the respondent assesses the utility that would result from each I&E reduction plan i (including the status q

uo) , and chooses the plan that would offer the highest utility

. When faced with k distinct plans defined by their attributes, the respondent will choose plan i if the anticipated utility from plan i exceeds that of all other k

- 1 plans. Drawing from (1), the respondent will choose plan i if
(2) (v( X i , D , Y

- F i ) + ε i ) ≥ (v( Xk , D , Y

- F k) + ε k)  k≠i

.If the ε i are assumed independently and identicall y drawn from a t ype I extreme value (Gumbel) distribution, the model may be estimated as a conditional logit model, as detailed b y Maddala (1983), Greene (2003) and others. This model is most commonly used when the respondent considers more than two options in each choice set (e.g., Plan A, Plan B, Neither Plan), and results in an econometric (empirical) e stimate of the sy stematic component of utility v( . ) , based on observed choices among different policy plans. Based on this estimate, one may calculate welfare measures (willingness to pay) following the well

- known methods of Hanemann (1984), as described by

Freeman (2003) and others. Following standard choice experiment methods (Adamowicz et al. 1998; Bennett and Blamey 2001), each respondent will consider questions including three potential choice options (i.e., Plan A, Plan B, Neither Plan)

—choosing the option that provides the highest utility as noted above. Following clear guidance from the literature, a “neither plan” or status quo option is alway s included in the visible choice set, to ensure that WTP measures are well

- defined (Louviere et al. 2000). EPA also anticipates that respondents will consider more than one choice question within the same survey , to increase information obtained from each respondent. This is standard practice within choice experiment and dichotomous choice contingent valuati on survey s (Poe et al. 1997; L ay

ton 2000). While respondents will be instructed to consider each choice question as independent of other choice questions, it is nonetheless standard practice within the literature to allow for the potential of correlation among questions answered within a single survey by a single respondent. That is, responses provided b y individual respondents may be correlated even though responses across different respondents are considered independent and identically distributed (Poe et al. 1997; L ay ton 2000; Train 1998). There are a variety

of approaches to such potential correlation. Following standard practice, EPA anticipates the estimation of a variety of models to assess their performance. Models to be assessed include rando m effects and random parameters (mixed) discrete choice models, now common in the stated preference literature (Greene 2003; McFadden and Train 2000; Poe et al. 1997; L ay ton 2000). Within such models, selected elements of the coefficient vector are assume d normally distributed across respondents, often with free correlation allowed among parameters (Greene 2002). If onl y the model intercept is assumed to include a random component, then a random effects model results. If both slope and intercept parameter s may vary across respondents, then a random parameters model is estimated. EPA anticipates thatsuch models will be estimated using standard maximum likelihood for mixed conditional logit

techniques, as described by Train (1998), Greene (2002) and others . Mixed logit model performance of alternative specifications will be assessed by EPA using standard statistical measures of model fit and convergence, as detailed by Greene (2002, 2003) and Train (1998).
Advantages of Choice Experiments Choice experiments following the random utility model outlined above are favored by many researchers over other variants of stated preference methodology (Adamowicz et al. 1998; Bennett and Blamey

2001), and may

be viewed as a “natural generalization of a binary discrete choice CV [contingent valuation]” (Bateman et al. 2002, p. 271). Advantages of choice experiments include a capacity to address choices over a wide array of potential policies, grounded in well

- developed random utility theory, and the similarity of the discrete choice context to familiar referendum or voting formats (Bennett and Blamey 2001). Compared to other ty pes of stated preference valuation, choice experiments are better able to measure the marginal value of changes in the characteristics o r attributes of environmental goods, and avoid response difficulties and biases (Bateman et al. 2002). For example, choice experiments may reduce the potential for ‘yea - saying’ and s ymbolic biases (Blamey et al. 1999; Mitchell and Carson 1989), as man y pa irs of multi

- attribute policy choices (e.g., Plan A, Plan B, Neither) will offer no clearly

superior choice for a respondent wishing to express solely sy mbolic environmental motivations. For similar reasons choice experiments may ameliorate protest respon ses (Bateman et al. 2002). An additional advantage of such methods is that they permit straightforward assessments of the impact of resource scope and scale on respondents’ choices. This will enable EPA to easily conduct scope tests and other assessments of the validity of survey responses (Bateman et al. 2002, p. 296

- 342). Finall y, such methods are well

- established in the stated preference literature (Bennett and Blamey 2001). Additional details of choice experiment methodology (also called choice mode ling) are provided b y Bennett and Blamey (2001), Adamowicz et al. (1998), Louviere et al. (2000) and man y other sources in the literature. An additional advantage of choice experiments in the present application is that they are commonly

applied to asses s WTP for ecological resource improvements of a ty pe quite similar to those at issue in the 316(b) policy case. Examples of the application of choice experiments to estimate WTP associated with changes in aquatic life and habitat include Hoehn et al. (200 4),Johnston et al. (2002b), and Opaluch et al. (1999), among others. EPA has drawn upon these and

other examples of successful choice experiment design to provide a basis for survey design in the present case. A final key

advantage of choice experiments in the present application is the ability to estimate respondents’ WTP for a wide range of different potential outcomes of 316(b) policies, differentiated b y their attributes. The proposed choice experiment survey versions will allow different respondents to choose among a wide variety of h yp othetical policy

options, some with larger and other with very small changes in the presented attributes (annual fish losses, long

- term fish populations, recreational and commercial catch, ecos ys tem condition, and hous ehold cost). That is, because the survey is to be implemented as a choice experiment survey , levels of attributes in choice scenarios will vary across respondents (Louviere et al. 2000). The experimental design will also explicitly allow for variation in baseline population and harvest levels, following standard practice in the literature (Louviere et al. 2000; Bateman et al. 2002). Aside from providing the capacity to estimate WTP for a wide range of policy outcomes, it also frees EPA from having to pr edetermine a single policy outcome for which WTP will be estimated. Given the potential biological uncertainty involved in the 316(b) policy case, the ability to estimate values for a wide range of potential outcomes is critical. The ability

to estimate W TP for a wide range of different policy outcomes is a fundamental propert y of the choice experiment method (Bateman et al. 2002; L ouviere et al. 2000; Adamowicz et al. 1998). For the purpose of stated preference survey implementation , EPA will use four ge ographic regions: Northeast, Southeast, Inland, and Pacific. The Northeast regional survey

is included in this I CR as Attachment 1. In addition, EPA will administer a national version of the survey that is included as Attachment 2. EPA emphasizes that the survey versions included in this ICR are for illustration only ; they are but two of what will ultimately be a large number of different survey versions covering a wide range of potential policy outcomes as described in Attachment 4. The experimental des ign (see below) will allow for survey versions showing a range of different baseline and resource improvement levels, where these levels are chosen to (almost certainl y) bound the “actual” levels. Given that there will almost certainl y be some biological uncertainty regarding the specifics of the “actual” baselines and improvements, the resulting valuation estimates will allow flexibility in estimating WTP for awide range of different circumstances. Additional details on the statistical (experimental) de sign

of the choice experiment is provided in later sections of this I CR.

Comment on Survey Preparation and Pretesting Following standard practice in the stated preference literature (Johnston et al. 1995; Desvousges and Smith 1988; Desvousges et al. 198 4; Mitchell and Carson 1989), all survey elements and methods were subjected to extensive development and pretesting in focus groups to ameliorate the potential for survey biases (cf. Mitchell and Carson 1989), and to ensure that respondents have a clear understanding of the policies and goods under consideration, such that informed choices may

be made that reflect respondents’ underl yi ng preferences. Following the guidance of Arrow et al. (1993), Johnston et al. (1995), and Mitchell and Carson (1989), foc us groups were used to ensure that respondents are aware of their budget constraints, the scope of the resource changes under consideration, and the availability of substitute environmental
resources. As noted above, survey

pretests included individual c ognitive interviews with survey respondents (Kaplowicz et al. 2004), and think

- aloud or verbal protocol analy ses (Schkade and Pay ne 1994). Individuals in these pretests completed draft survey questionnaires and provided comments and feedback about the sur vey format and content, their interpretations of the questions, and other issues relevant to stated preference estimation. Based on their responses, EPA made improvements to the survey questionnaire. Of particular emphasis in these survey pretests was te sting for the presence of potential biases including h ypothetical bias, strategic bias, sy mbolic (warm glow) bias, framing effects, embedding biases, methodological misspecification, and protest responses (Mitchell and Carson 1989). Based on focus group a nd cognitive interview responses, EPA made various improvements to the survey questionnaire including changes to ameliorate and minimize these biases in the final survey instrument. Results from focus groups and cognitive interviews provided evidence that respondents answer the stated preference survey

in ways appropriate for stated preference WTP estimation, and that their responses generall y do not reflect the biases noted above. The number of focus groups used in survey design, seven (excluding the 12 focus groups conducted for the Phase III survey), exceeds the number of focus groups used in ty picalapplications of stated preference valuation. Moreover, EPA incorporated cognitive interviews as

detailed b y Kaplowicz et al. (2004). We note that the cur rent survey

instrument is built upon an earlier version that was peer reviewed in January 2006 (Versar 2006) and it incorporates recommendations received from that peer review panel. Given this extensive effort in survey design

— applying the most state

- of

- the

- art methods available in the literature

—EPA believes that survey design far exceeds standards that are ty pical in the published literature. The details of focus groups conducted for the previous Phase III survey are discussed b y EPA in a prior ICR (#21 55.01 ).
Econometric Specification Based on prior focus groups, expert review, and attributes of the policies under consideration, EPA anticipates that four attributes will be incorporated in the vector of variables describing attributes of an I&E reducti on plan (vector X i ), in addition to the attribute characterizing unavoidable household cost F i . 3 These attributes will characterize the annual reduction in I&E losses (

x 1 ), anticipated effects on fish populations (all fish) ( x2 ), anticipated effects on c ommercial fish populations ( x 3 ), and anticipated effects on aquatic ecos ys tem condition (

x 4 ). These variables will allow respondents’ choices to reveal the potential impact of both annual fish losses and long

- term population effects on utility . Based on results of focus groups and expert opinion, these will be presented as averages across identified aggregate species groups. The survey will also allow for changes in baseline population levels, to assess whether WTP depends on the “starting point” of fish populations. Although the literature offers no firm guidance regarding the choice of specific functional forms for v(

. ) within choice experiment estimation, in practice, linear forms are often used (Johnston et al. 2003b), with some researchers applying more flexible (e.g., quadratic) forms (Cummings et al. 1994). Standard linear forms are anticipated as the simplest form to be estimated by

EPA, from which more flexible functional forms (able to capture interactions among model variables) will be derived and compared. Anticipated extensions to the simple linear model include more fully

- flexible forms that allow for s ys tematic variations in slope and

3 EPA plans to complete focus group testing of the instrument before the second Federal Register notice of this information collection request. The inclusion of all four attributes is an important aspect of focus group testing. If focus groups find this number co gnitively challenging, the number will be reduced, and if cognitive issues are minimal as identified by randomly selected focus group participants, all four will remain.intercept coefficients associated with demographic or other attributes of respondents. Such

variations may be incorporated b y appending the simple linear specification with quadratic interactions between variables in vector D and the variables X i and F i (cf. Johnston et al. 2003b). One may also incorporate quadratic interactions between policy attributes X i and F i , (cf. Johnston et al. 2002b). Such quadratic extensions of the basic linear model allow for additional flexibility in modeling the relationship between policy attributes (including cost) and utility , as suggested b y Hoehn (1991) and Cummings et a l. (1994). EPA anticipates estimating both simple linear specifications, as well as more full y - flexible quadratic specifications following Hoehn (1991) and Cummings et al. (1994), to identify those models which provide the most satisfactory statistical fi t to the data and correspondence to theory . EPA anticipates estimating all models within the mixed logit framework outlined above. Model fit will be assessed following standard practice in the literature (e.g., Greene 2003; Maddala 1983). Linear and qua dratic functional forms discussed here, as they are common practice in the literature, are presented and discussed in many existing sources (e.g., Hoehn 1991, Cummings et al. 1994, Johnston et al. 1999, and Johnston et al. 2003b). For example, for each ch oice occasion, the respondent may choose Option A, Option B, or Neither, where “neither” is characterized by 0 values for all attributes (except Baseline population levels). Assuming that the model is estimated using a standard approximation for the observ able component of utility , an econometric specification of the desired model (within the overall multinomial logit model) might appear as: v(  ) =  0 +  1 (Fish Saved) +  2 ( Change in Populations of All Fish ) +  3 (Change in Commercial Fish Populations ) +  4 (Change in Condition of Aquatic Ecosystem) +  5 (Cost) +  6 (Fish Saved)(Baseline) +  7 ( Change in Population s of All Fish )(Baseline) + 

8 (Change in Commercial Fish Populations )(Baseline) + 

9 (Change in Aquatic Ecosystem)(Baseline) +  10 (Cost)(Baseline) +  1 1 (Fish Saved)(

Change in Population s of All Fish ) +  12 (Fish Saved)(Change in Commercial Fish Populations ) +  13 (Fish Saved)(Change in Aquatic Ecosystem) + 

14 ( Change in Population s of All Fish )(Change in Commercial Fish Populations ) +  15 ( Change in Population s of All Fish )(Change in Aquatic Ecosystem) +  16 (Change in Commercial Fish Populations )(Change in Aquatic Ecosystem)

Main effects are in bold. Interactions are in italics. This sample specification

—one of man y to be estimated b y EPA

— allows o ne to estimate the relative “main effects” of policy attributes (annual reduction in I&E losses, long

- term effects on populations of all fish, long

- term effects oncommercial fish populations) on utility , effects on aquatic ecosy stem condition, as well as

interactions between these main effects. This specification also allows EPA to assess the impact of baseline fish populations on the marginal value of changes in other model attributes. In sum, specifications such as this allow WTP to be estimated for a wide

- range of potential policy outcomes, and allow EPA to test for a wide

- range of main effects and interactions within the utility function of respondents. Such flexible utility specifications for stated preference estimation are recommended by numerous sources in the literature, including Johnston et al. (2002b), Hoehn (1991), and Cummings et al. (1994), and follow standard practice in choice modeling outlined b y Lo uviere et al. (2000) and others.
Experimental Design Experimental design for the choice experiment survey s will follow established practices. Fractional factorial design will be used to construct choice questions with an orthogonal array of attribute levels, with questions randomly divided among distinct survey versions (L ouviere et al. 2000 ). Based on standard choice experiment experimental design procedures (Louviere et al. 2000), the number of questions and survey versions will be determined by , among other factors: a) the number of attributes in the final experimental design and complexi ty of questions, b) the extent to which estimation of interactions and higher

- level effects is desired, and c) pretests revealing the number of choice experiment questions that respondents are willing/able to answer in a single survey

session, and the numb er of attributes that may be varied within each question while maintaining respondents’ ability to make appropriate neoclassical tradeoffs. Based on the models proposed above and recommendations in the literature, EPA anticipates an experimental design that allows for an ability to estimate main effects, quadratic effects, and two

- way

interactions between policy attributes (L ouviere et al. 2000). Choice sets (Bennett and Blamey

2001), including variable level selection, will be designed b y EPA based on the goal of illustrating realistic policy scenarios that “span the range over which we expect respondents to have preferences, and/or are practically achievable” (Bateman et al. 2002, p. 259), following guidance in the literature. This includes guidance wi th regard to the statistical implications of choice set design (Hanemann and Kanninen 1999) and the role of focus groups in developing appropriate choice sets (Bennett and Blamey 2001).Based on these guiding principles, the following experimental design framework is

proposed b y EPA. The experimental design will be conducted b y Abt Associates Inc. The experimental design will allow for both main effects and selected interactions to be e fficientl y estimated, based on a choice experiment framework. For a more detailed discussion of the experimental design, refer to Attachment 4. Each treatment (survey

question) includes two choice Options (A and B), characterized by four attributes and a cost variable that vary across the two choice options (Commercial Fish Populations, Fish Populations (all fish), Fish Saved per Year, Condition of Aquatic Ecos ys tems, and Increase in Cost of Living of Your Household). Hence, there are a total of ten attributes for each treatment. Based on focus groups and pretests, and guided b y realistic ranges of attribute outcomes, EPA allows for three different potential levels for Commercial Fish Populations, Fish Population s (all fish) , Fish Saved per Year, and Cond ition of Aquatic Ecosy stems, and allows for six different levels of annual Household Cost for the regional or national choice questions. The survey versions also include two separate treatments of location, or geographic scope; the first version refers to policies only within the respondent’s region and the second version refers to national policies. The number of combinations for each attribute may be summarized as follows:
 Location A , LocationB (2 levels)  Commercial Fish Populations A , Commercial Fish Populations B (3 levels)  Fish Population s (all fish) A , Fish Population s (all fish) B (3 levels)  Fish Saved per YearA , Fish Saved per YearB (3 levels)  Condition of Aquatic Ecosy stems A , Condition of Aquatic Ecos ys tems B (3 levels)  CostA, CostB (6 levels) Be yond the levels specified above, each question will include a “no policy” option, characterized by baseline levels for each attribute including a household cost of $0. Following standard practice, EPA constrained the design somewhat in res ponse to findings in seven focus groups and the prior literature. For example, the focus groups showed that respondents react negativel y and often protest when offered choices in which one option dominates the other in all attributes. Given that such cho ices provide negligible statistical information compared to choices involving non

- dominant/dominated pairs, they are t ypically avoided in choice experiment statistical designs. For example, Hensher and Barnard (1990)recommend eliminating profiles includin g dominating or dominated profiles, because such

profiles generall y provide no useful information. Following this guidance, EPA constrained the design to eliminate such dominant/dominating pairs. EPA also constrained the design to eliminate the possibili ty

of pairs in which, when looking across two options, one of the options offers both a greater reduction in fish losses and a smaller increase in the population. The elimination of such nonsensical (or non

- credible) pairs is common practice, and is done to avoid protest bids and confusion among respondents (Bateman et al. 2002). The resulting experimental design is characterized by 72 unique A vs. B option pairs, where attribute levels for option A and B differ across each of the pairs. Each pair represe nts a unique choice modeling question

— with a unique set of attribute levels distinguishing options A and B. Following standard practice for mail survey s, these questions will be randoml y assigned to survey respondents, with each respondent considering thr ee questions.
Information Provision According to Arrow et al. (1993, p. 4605), if “surveys are to elicit useful information about willingness to pay , respondents must understand exactly what it is they

are being asked to value.” It is also well known that the provided information can influence WTP estimates derived from stated preference survey instruments and that respondents must be provided with sufficient information to make an informed assessment of policy impacts on utility

(e.g., Bergstrom and St oll 1989; Bergstrom et al. 1989; Hoehn and Randall 2002). As stated clearly b y Bateman et al. (2002, p. 122), “[d]escribing the good and the policy context of interest may require a combination of textual information, photographs, drawings, maps, charts and graphs. ...[V]isual aids are helpful way s of conveying complex information...while simultaneously enhancing respondents’ attention and interest.” Given that many respondents may

not be fully

familiar with the details of programs to reduce I&E mortality los ses and potential impacts on aquatic life, the survey will include introductory figures to aid respondents’ comprehension of the goods and policies addressed b y the survey instrument, and to encourage appropriate neoclassical tradeoffs in responding to cho ice experiment questions. Following this guidance of Bateman et al. (2002) and prior examples of Opaluch et al. (1993) and Johnston et al. (2002a), among others, EPA extensively pretested all graphics used in the draft mail survey , to ensure that these gr aphical elements were not prejudicial, and that theydid not bias responses. Graphics judged to be prejudicial or confusing to respondents during the

seven focus groups and cognitive interviews were revised or replaced. EPA acknowledges that certain t ypes of graphics can be prejudicial in certain contexts

— and hence all graphical elements were pretested extensively . EPA found that focus group respondents endorsed the use of graphics in the survey

booklet and indicated that they helped them to visualize how fish are entrained and impinged, technological solutions, facilities locations, and ecosy stem effects. Participants made such statements as , “Yeah, I’d rather have them” and “I like on page 2 the graph and illustration because the adage a picture is wort h a thousand words”. EPA also emphasizes that there is no precedent or support in the literature for the total elimination of graphics in survey

instruments. To the contrary , the literature explicitly indicates that pictures and graphics may

be necessary and useful components of survey instruments in many

cases (Bateman et al. 2002). EPA highlights that numerous peer

- reviewed surveys described in the literature include pictures and graphics both in survey instruments and in introductory materials such as slide shows. For example, see Horne et al. (2005), Ready et al. (1995), Powe and Bateman (2004), Duke and I lvento (2004), Opaluch et al. (1993), Johnston et al. (1999, 2002a, 2002b), and Mazzotta et al. (2002). Bateman et al. (2002) also includes exampl es of various ty pes of survey

materials including pictures and graphical elements.
Amelioration of Hypothetical Bias EPA considers the amelioration of h ypothetical bias to be a paramount concern in survey design. However, the agency acknowledges

—

based on prior evidence from the literature

—that hy pothetical bias is not unavoidable. For example, not all research finds evidence of hy pothetical bias in stated preference valuation (Champ and Bishop 2001; Smith and Mansfield 1998; Vossler and Kerkvliet 2003; Johannesson 1997), and some shows that hy pothetical bias may be ameliorated using cheap

- talk, certainty adjustments, or other mechanisms (Champ et al. 1997; Champ et al. 2004; Cummings and Taylor 1999; L oomis et al. 1996). To obtain reliable estimates o f WTP, the Agency tested and designed all survey elements to promote incentive compatible preference elicitation mechanisms

. Incentive compatible stated preference survey s provide no incentive for non

- truthful preference revelation (Carson and Groves 2007) . The literature is clear regarding the importance of incentive compatibility in stated preference value elicitation and the role of both question format and scenario consequentiality inensuring this propert y (C arson et al. 2000; Carson and Groves 2007; C ollins and Vossler 2009;

Herriges et al. 2010; Johnston 2006; Vossler and Evans 2009). It has been established that referendum

- ty

pe stated preference choices are incentive compatible given that certain conditions are met, including the condition that resp onses are believed b y respondents to be consequential, or potentially

influencing public policy decisions (Carson and Groves 2007; Herriges et al. 2010). The survey

is explicitly

designed to emphasize the importance of the budget constraint and program co st. For example, the survey asks respondents to compare protecting aquatic
ecos ys tems to other policy issues which the government could potentially ask households to pay costs. The survey

itself includes explicit reminders of program cost and the budget constraint. The survey

has also been explicitly

designed to maximize the consequentiality of choice experiment questions, thereb y maximizing incentive compatibility (i.e., reducing strategic and hy pothetical biases), following clear guidance of Carson et al. (2000). Elements specificall y designed to maximize consequentiality include: a) explicitly

mentioning that this survey is associated with assessment of proposed policies that are being considered, b) numerous details provided in the survey

concerning specifics of the proposed policies, and c) emphasis that the ty pe of policy

enacted will depend in part on survey results and that their vote is important. Johnston and Joglekar (2005) show the capacity of such information to eliminate hy pothetical bias in choice

- base d stated preference WTP estimation. Focus groups and cognitive interviews indicated that respondents viewed choices as consequential, that they

considered their budget constraints when responding to all questions, and that they

would answer the same way were similar questions asked in a binding referendum. When asked if they

thought about the program cost in the same way as “money

coming out of their pocket,” the vast majority of focus group and interview respondents indicated that they treated program c osts the same way that they would have if there were actual money consequences. For example, respondents made statements such as “No. [M y vote] would have been the same actuall y” and “If I believed that it was gonna affect regulations, Ithink I would hav e voted the exact same way

. ” EPA does not anticipate significant h ypothetical bias in the proposed survey based on focus group results. Focus groups respondents took the survey questions seriously

and indicated that they though that their choices would actuall y influence policy . Regarding the potential useof cheap talk mechanisms or other devices to further address the potential for hy pothetical bias, the Agency

emphasizes that the literature is mixed as to their performance. For example, the seminal wor

k by Cummings and Taylor (1999) shows that cheap talk is able to reduce hy pothetical biases. Similar results are shown by Aadland and Caplan (2003). However, other authors (e.g., Cummings et al. 1995; L ist 2001; Brown et al. 2003) find that a cheap talk script is only effective under certain circumstances, and for certain ty pes of respondents. For example, Cummings et al. (1995) find that a relativel y short cheap talk script actuall y worsens hy pothetical bias, while a longer script appears to ameliorate bias. Brown et al. (2003) finds cheap talk onl y effective at higher bid amounts

— a result mirrored b y Murphy et al. (2004). Still other authors find no effect of cheap talk, including Poe et al. (2002). Given the clearl y mixed experiences with such mechanisms, EPA is not convinced that cheap talk scripts are likel y to provide a panacea for h ypothetical bias in the present case

— although they appear to reduce bias in a limited set of circumstances

– and cheap talk is not included in the survey

.
Ameliorati on of Symbolic Biases and W arm - Glow Effects Following clear guidance of Arrow et al. (1993) and others, EPA has taken repeated steps to ensure that survey

responses reflect the value of the affected fish resources only , and do not reflect sy mbolic or warm glow concerns (Mitchell and Carson 1989). Following explicit guidance of the NOAA Blue Ribbon Panel on Contingent Valuation (Arrow et al. 1993, p. 4609), EPA has explicitly

designed all elements of the survey to “deflect the general ‘warm glow’ of giving or the dislike of ‘big business’ away from the specific program that is being valued.” This was done in a variety of way s, based on prior examples in the literature, such as asking respondents to reflect on importance of attributes before making selection s, and using a pay ment vehicle that doesn’t raise trust issues (cost of living increase rather than an electric bill increase). The focus group and cognitive interview results indicated that most participants answered the choice questions based on the effe cts discussed in the survey , not on a desire to help the environment in general. The survey

includes clear language to instruct respondents only to consider the specific attributes in the survey , and not to base answers on broader environmental concerns i ncluding the statement that:“ Scientists expect that effects on the environment and economy not shown explicity will

be small. For example, studies of industry suggest that effects on employment will be close to zero.” This is also consistent with the st atement from Arrow et al. (1993) that a referendum

- ty pe format may limit the warm

- glow effect. Some focus group participants indicated that they were inclined to support environmental causes and would like “to do a good thing” but still considered the cos t and effects under the policy options. For example, respondents stated, “[...] if we can do something to help as long as the price is right, then do it” and “I feel if it’s going to be benefit everyone and be better for the econom y, I’m OK with paying a little bit more. ” This evidence notwithstanding, EPA believes that it is important to include follow

- up questions to ensure that responses do not reflect sy mbolic biases. Question 9 in the survey instrument

— which addresses the rationale for choice responses given earlier in the draft survey

— explicitly

tests for the presence of s ymbolic or warm

- glow biases. Follow

- up questions such as these are common in stated preference survey instruments, to assess the underl yi ng reasons for the observed valuation respons es (e.g., Mitchell and Carson 1984).
Assessing Scope Sensitivity The NOAA Blue Ribbon Panel on Contingent Valuation (CV) (Arrow et al. 1993, p. 4605) states clearl y that if “surveys are to elicit useful information about willingness to pay, respondents mu st understand exactly what it is they

are being asked to value (or vote upon)...” They further indicate that survey s providing “sketchy details” about the results of proposed policies call “into question the estimates derived there from,” and hence suggest a high degree of detail and richness in the descriptions of scenarios. Similar guidance is provided by other key sources in the CV literature (e.g., Mitchell and Carson 1989; L ouviere et al. 2000). Among the reasons for this guidance is that such descript ions tend to encourage appropriate framing and sensitivity

to scope. Following Arrow et al. (1993), Mitchell and Carson (1989), and others, while noting the clear limitations in scope tests discussed by Heberlein et al. (2005), EPA believes that it is impo rtant that survey

responses in this case show sensitivity to scope. This is one of the primary reasons for the use of the choice experiment methodology , which is better able to capture WTP differentials related to changes in resource scope (Bateman et al. 2002). Unlike open

- ended questions, in which scope insensitivity is a primary

concern, EPA emphasizes that choiceexperiments generall y have shown much less difficulty with respondents reacting appropriatel y

to the scope and scale of resource changes. M oreover, as clearl y noted b y Bennett and Blamey (2001, p. 231), “internal scope tests are automatically available from the results of a [choice modeling] exercise.” That is, within choice experiments, sensitivity to scope is indicated by

the statistical significance and sign of parameter estimates associated with program attributes (Bennett and Blamey

2001). Internal scope sensitivity will therefore be assessed through model results for the variables Commercial Fish Populations, Fish Populations (all fis h), Fish Saved per Year, and Condition of Aquatic Ecos ys tems. Statistical significance of these variables

— along with a positive sign

—indicates that respondents, on average, are more likely to choose plans with larger quantities of these variables. In ad dition to internal scope tests implicit in all choice experiment statistical anal ys is, EPA will also conduct external scope tests (cf. Giraud et al. 1999). The primary difference between internal and external tests is that the former assess sensitivity to scope across choices of a single respondent, while the latter involves split

- sample assessments across different respondents. Within a choice modeling context, external scope tests are generall y considered “stronger,” although also more likely to be conf

ounded by

differences in the implied choice frame (Bennett and Blamey 2001). A variet y of options for external scope tests exist, depending on the structure of the stated choice questions under consideration. In the present case, attribute

- by

- attribute ex ternal scope tests will be conducted over a split sub

- sample of respondents considering a specific set of choices, with all held constant across the considered choices except the scope of the attribute for which the test is to be conducted. For example, t o conduct an external scope test for reductions in annual fish losses, one would consider a set of choices that is identical over two respondent groups, except that one considers a choice with a greater reduction in fish losses. Assessing the choices over this split sample allows for an external test of scope. To illustrate this test, consider the following styl ized choice between Option A and Option B. The generic labels “Level 0”, “Level 1”, and “Level 2” are used to denote attribute levels, where for all attributes Level 2 > Level 1 > Level 0.Table B2: Illustration of an External Scope Test

Variable Option A Option B Fish Saved per Year Sample 1: Fish Saved Level 1 Sample 2: Fish Saved Level 2 Fish Saved Level 0

Commercial Fish Populations Commercial Fish Populations Level 0 Commercial Fish Populations Level 0 Fish Populations (all fish) Population (all fish) Level 0 Population (all fish) Level 0 Condition of Aquatic Ecosystems Aquatic Ecosystem Condition Level 0 Aquatic Ecosystem Condition Level 0 Increase in Cost of Living for Your Household Cost Level 1 Cost Level 0

In the above example, only Fish Saved per Year and Cost vary across the choice options. Because both Fish Saved per Year (at Level 1 and L evel 2) and Cost are higher in Option A than in Option B, neither option is dominant. In the illustrated split

- sampl e test, respondent sample 1 views the choice with Fish Saved per Year at Level 1, while respondent sample 2 views an otherwise identical choice with Fish Saved per Year at Level 2, where Level 2 > Level 1. If responses are externall y sensitive to scope in Fish Saved per Year, this will manifest in a greater proportion of sample 2 respondents choosing Option A than sample 1 respondents. This hy pothesis may

be easil y assessed using a test of equal proportions across the two sub

- samples, and provides a simpl e attribute

- by

- attribute test of external scope. Analogous tests may be conducted for all attributes within the choice experiment design, using parallel methods. EPA emphasizes that the formal applicability of the above

- noted scope test is contingent upon the specific choice frame implied by levels of other attributes in the choice question. This is a characteristic of nearl y all external scope tests applied in choice experiment frameworks (Bennett and Blamey

2001). Split

- sample tests such as those propos ed above often require the addition of question versions to the experimental design, to accommodate the specific structural needs of the attribute

- by

- attribute external scope test. Otherwise, confounding effects of other varying attributes (including demo graphic information) can render results of scope tests ambiguous. In the present case, the proposed tests would require the addition of up to six unique questionversions to the experimental design, enabling scope tests for the three non

- cost attributes w ithin

the 316(b) choice experiment scenarios. If scope tests in additional question frames are desired (e.g., the same scope test illustrated above, but given Level 1 for commercial fish populations, fish population (all fish), and aquatic ecos ys tem condi tion attributes), still additional question versions would be added. While small numbers of questions added to the experimental design should have minimal impacts on overall efficiency (e.g., orthogonality of the design), larger numbers may

have a more si gnificant impact. Hence, given constraints on the total number of survey respondents, there is a potential empirical tradeoff between the number of external scope tests that may

be conducted and the efficiency of the experimental design and statistical an aly sis.
Communicating Uncertainty to Respondents EPA believes that the role of risk and uncertainty is an important issue to be addressed in the development of benefits estimates, and points out that the literature provides numerous examples of cases in which appropriate survey design, including focus groups, was used to successfull y address such concerns. For example, as stated b y Desvousges et al. (1984), “using contingent valuation to estimate the benefits of hazardous waste management regulations requ ires detailed information on how and the extent to which respondents understand risk (or probability) and how government regulatory actions might change it... Using focus groups helped make this determination...” EPA also emphasizes that all regulatory analys es involve uncertaint y of some t ype (Boardman et al. 2001). The ecological outcome of I&E reductions is subject to considerable uncertainty . EPA believes that it is important that survey respondents be aware of this uncertainty , and that their responses reflect the knowledge that the resource changes reflected in the survey are scientific estimates. However, EPA is also aware of the clear advice from the choice modeling literature (e.g., Bennett and Blamey 2001; Louviere et al. 2000) to avoid cognitive burden on respondents. Hence, the proposed survey materials clearl y indicate the uncertainty involved with the described resource changes in choice modeling scenarios, yet do so in a way designed to minimize cognitive burden. For example, prior to answeri ng choice experiment questions, respondents are told: “Although scientists can predict the number of fish saved each year, the effect on fish populations is uncertain. This is because scientists do not know the total number of fish inNortheast waters and because many factors

– such as cooling water use, fishing, pollution

and water temperature

– affect fish populations.” This statement clearl y indicates the uncertainty involved with scientific estimates of the outcomes of I&E regulations. This is follow ed by a further reminder of uncertainty : “Depending on the type of technology and other factors, effects on fish and ecosystems may be different

– even if the annual reduction in fish losses is similar.” Focus groups and cognitive interviews participants understood that the ecological changes described in the survey

were uncertain, and most participants were comfortable making decisions in the presence of this uncertainty. Their responses indicated that they understood this uncertaint y based on the information presented in the introductory material and considered it when evaluating policy

options. Respondents made such statements as: “My guess is that it did come from studies but I have a health y dose of skepticism about the accuracy of it. I don’t think it ’s been in an y way skewed purposefully , but I know that this is a best guess, reasonable guess perhaps” , “it shows me that they are being honest for the most part. You know, you can't obviously be accurate on every

thing, but this is a kind of a best guess ” , “[...] They had more numbers on the commercial fish population. The rest was more of a guesstimat e”, and “you don’t know the exact number and nobody knows

. ” In previous focus groups conducted for the Phase III survey , EPA tested alternative versions of the Phase III survey instrument in which choice experiment attributes were presented as 90% confidence ranges, rather than as point estimates. Focus group respondents were explicitly asked whether the ranges wer e helpful in understanding the uncertainty of estimates presented in the choice question or whether they were a source of confusion. Seven out of the eight respondents interviewed on that occasion indicated that the use of ranges was more confusing than the use of point estimates. Furthermore, respondents were comfortable making decisions in the presence of this uncertaint y.
5(c) Reporting Results
The results of the survey

will be made public as part of the benefits anal ys is for the 316(b) regulation f

or existing facilities. Provided information will include summary statistics for the survey

data, extensive documentation for the statistical analy sis, and a detailed description ofthe final results. The survey data will be released only after it has be en thoroughl y vetted to ensure that all potentiall y identifying information has been removed.REFERENCES

Aadland, D., and A.J. Caplan. 2003. “Willingness to Pay for Curbside Recycling with Detection and Mitigation of Hy

pothetical Bias.” American Journal of Agricultural Economics 85(2): 492

- 502. Adamowicz, W., P. Boxall, M. Williams, and J. Louviere. 1998. “Stated Preference Approaches for Measuring Passive Use Values: Choice Experiments and Contingent Valuation.” American Journal of Agricultural Economics 80(1): 64

- 75. Akter, S., R. Brower, L. Brander , and P. Van Beukering. 2009. “ Respondent Uncertaint y in a Contingent Market for Carbon Offsets. ” Ecological Economics 68(6): 1858

- 1863. Arrow, K. , R. Solow, E. Leamer, P. Portney , R. Rander, and H. Schuman. 1993. “Report of the NOAA Panel on Contingent Valuation.” Federal Register 58 (10): 4602

- 4614. Bateman, I.J., R.T. Carson, B. Day , M. Hanemann, N. Hanley , T. Hett, M. Jones

- Lee, G. Loomes, S. Mourato, E. Ozdemiroglu, D.W. Pierce, R. Sugden, and J. Swanson. 2002. Economic Valuation with Stated Preference Surveys: A Manual . Northampton, MA: Edward Elgar. Bennett, J. and R. Blamey, eds. 2001. The Choice Modelling Approach to Environmental Valuation . Northampton, MA: Edward Elgar. Bergstrom, J.C. and J.R. Stol l. 1989. “Aplication of experimental economics concepts and precepts to CVM field survey procedures.” Western Journal of Agricultural Economics 14(1): 98

- 109. Bergstrom, J.C., J.R. Stoll, and A. Randall. 1989. “Information effects in contingent markets.” A merican Journal of Agricultural Economics 71(3): 685

- 691. Besedin, Elena, Robert Johnston, Matthew Ranson, and Jenny Ahlen, Abt Associates Inc. 2005. “Findings from 2005 Focus Groups Conducted Under EPA ICR #2155.01.” Memo to Erik Helm, U.S. EPA/OW, Octob er 18, 2005. See docket for EPA ICR #2155.02 Blamey , R.K., J.W. Bennett, and M.D. Morrison. 1999. “Yea

- saying in Contingent Valuation Survey s.” Land Economics 75: 126

- 141. Boardman, A.E., D.H. Greenberg, A.R. Vining, and D.L . Weimer. 2001. Cost

-Benefit An alysis: Concepts and Practice , 2nd edition. Upper Saddle River, NJ: Prentice Hall.Bo yl e, K.J. 2003

. “ Contingent valuation in practice.” In A Primer on N onmarket V

aluation

. Edited by P.A. Champ, K.J. Boyle, and T.C. Brown, Kluwer Academic Publishers. Brown, T. C., I. Ajzen, and D. Hrubes. 2003. “Further Tests of Entreaties to Avoid Hy pothetical Bias in Referendum Contingent Valuation.” Journal of Environmental Economics and Management 46(2): 353

- 361. Bunch, D.S., and R.R. Batsell. 1989. “A Monte Carlo Comparison of Estimators for the Multinomial L ogit Model.” Journal of Marketing Research 26: 56

- 68. Cameron, T.A., and D.D. Huppert. 1989. “OLS versus MLEstimation of Non

- market Resource Values with Pay

ment Card Interval Data.” Journ al of Environmental Economics and Management 17: 230

- 246. Carson, R.T., and T. Groves. 2007. “Incentives and informational properties of preference questions.” Environmental and Resource Economics 37(1): 181

- 210. Carson, R.T., T. Groves, and M.J. Machina. 2000. “Incentive and Informational Properties of Preference Questions.” Working Paper, Department of Economics, University of California, San Diego. Champ P.A., and R.C. Bishop. 2001. “Donation Pay ment Mechanisms and Contingent Valuation: An Empirical Study of Hy pothetical Bias.” Environmental and Resource Economics 19(4): 383

- 402. Champ, P.A., R.C. Bishop, T.C. Brown, and D.W. McCollum. 1997. “Using Donation Mechanisms to Value Non

- use Benefits from Public Goods.” Journal of Environmental Economics and Management 33(2): 151

- 162. Champ, P.A., R. Moore, and R. C. Bishop. 2004. “Hy pothetical Bias: The Mitigating Effects of Certainty Questions and Cheap Talk.” Selected paper prepared for presentation at the American Agricultural Economics Association Annual Meeting, Denver, Colorado. Champ, P.A., R. Moore , and R.C. Bishop. 2009. “ A Comparison of Approaches to Mitigate Hy pothetical Bias. ” Agricultural and Resource Economics Review 38(2): 166

- 180. Collins, J.P., and C.A. Vossler. 2009. “Incentive compatibility tests of choice experiment value elicitation questions.” Journal of Environmental Economics and Management 58(2): 226

- 235. Croke, K., R.G. Fabian, and G. Brenniman. 1986. “Estimating the Value of Improved Water Quality in an Urban River Sy stem.” Journal of Environmental Systems 16(1): 13

- 24.Cronin, F.J. 1982. “Valuing Nonmarket Goods Through Contingent Markets.” Pacific Northwest

Laboratory , PNL 4255, Richland, WA. Cummings, R.G., and G.W. Harrison. 1995. “The Measurement and Decomposition of Non

- use Values: A Critical Review.” Environmental and Resource Economics 5: 225

- 247. Cummings, R. G., G.W. Harrison, and L. L. Osborne. 1995. “Can the Bias of Contingent Valuation Survey s Be Reduced?” Economics working paper, Columbia, SC: Division of Research, College of Business Administration, Univ. of South Carolina. Cummings, R.G., P.T. Ganderton, and T. McGuckin. 1994. “Substitution Effects in CVM Values.” American Journal of Agricultural Economics 76(2): 205

- 214. Cummings, R.G., and L .O. Taylor. 1999. “Unbiased Value Estimates for Environmental Goods: A Cheap Talk Design for the Contingent Valuation Method.” American Economic Review 89(3): 649

- 665. Desvousges, W.H., and V.K Smith. 1988. “Focus Groups and Risk Communication: the S cience of L istening to Data.” Risk Analysis 8: 479

- 484. Desvousges, W.H., V.K. Smith, D.H. Brown, and D.K. Pate. 1984. “The Role of Focus Groups in Designing a Contingent Valuation Survey to Measure the Benefits of Hazardous Waste Management Regulations.” Research Triangle Institute: Research Triangle Park, NC. Dillman, D.A. 2000. Mail and Internet Surveys: The Tailored Design Method. New York: John Wiley and Sons. Duke, J.M., and T.W. I lvento. 2004. “A Conjoint Analy sis of Public Preferences for Agricultural Land Preservation.” Agricultural and Resource Economics Review 33(2): 209

- 219. Entergy Corp. v. Riverkeeper In c. , 129 S. Ct. 1498, 150 5 (2009) Freeman, A.M., III. 2003. The Measurement of Environmental and Resource Values: Theory and Methods . Washington, DC: Resources for the Future. Giraud, K.L., J.B. Loomis, and R.L . Johnson. 1999. “Internal and external scope in willingness

to
- pay estimates for threatened and endangered wildlife.” Journal of Environmental Management 56: 221

- 229. Greene, W.H. 2002. NLOGIT Version 3.0 Reference Guide . Plainview, NY: Econometric Software, Inc. Greene, W.H. 2003. Econometric Analysis . 5th ed., Prentice Hall, Upper Saddle River, NJ.Haab, T.C. , and K.E. McConnell. 2002. Valuing Environmental and Natural Resources: The

Econometrics of Non

- market Valuation. Cheltenham, UK: Edward Elgar. Hanemann, W.M. 1984. “Welfare Evaluations in Contingent Valuation Experiments with Discrete Responses.” American Journal of Agricultural Economics 66(3): 332

- 41. Hanemann, W.M., and B. Kanninen. 1999. “The Statistical Analy sis of Discrete

- Response CV Data.” In Valuing Environmental Preferences: Theory and Practice of the Contingent Valuation Method in the US, EU, and Developing Countries. Edited by I .J. Bateman and K.G. Willis, Oxford University Press, Oxford, UK. Hanley , N., S . Colombo, D. Tinch, A. Black, and A. Aftab. 2006 a . " Estimating the benefits of water qualit y improvements under the Water Framework Direct ive: are benefits transferable? ," European Review of Agricultural Economics 33(3):391

- 413. Hanley , N., R. E. Wright, and B. Alvarez

- Farizo. 2006 b

. “ Estimating the Economic Value of Improvements in River Ecology using Choice Experiments: An Application to the Water Framework Directive. ” Journal of Environmental Management 78(2):183

- 193. Heberlein, T.A., M.A. Wilson, R.C. Bishop, and N.C. Schaeffer. 2005. “Rethinking the Scope Test as a Criterion in C ontingent Valuation.” Journal of Environmental Economics and Management 50(1): 1

- 22. Heckman, J. J. 1979. “Sample Selection Bias as a Specification Error.” Econometrica 47(1): 153

- 161. Hensher, D. A., and Barnard, P. O. (1990). "The Orthogonalit y Issue in Stated Choice Designs." In Fischer, M., Nijkamp, P., and Papageorgiou, Y. (eds.), Spatial Choices and Processes. North

- Holland, Amsterdam, 265

- 278. Herriges, J., C. Kling, C. Lieu, and J. Tobias. 2010. “What are the consequences of consequentiality?” Journ al of Environmental Economics and Management 59(1): 67

- 81. Hoehn, J. P. 1991. “Valuing the Multidimensional Impacts of Environmental Policy : Theory and Methods.” American Journal of Agricultural Economics 73(2): 289

- 299. Hoehn, J.P., F. Lupi, and M.D. Kap lowitz. 2004. Internet

- Based Stated Choice Experiments in Ecosy stem Mitigation: Methods to Control Decision Heuristics and Biases. In Proceedings of Valuation of Ecological Benefits: Improving the Science Behind Policy Decisions , a workshop sponsored b y the US EPA National Center for Environmental Economics and the National Center for Environmental Research.Hoehn, J.P., and A. Randall. 2002. “The Effect of Resource Quality Information on Resource Injury Perceptions and Contingent Values.” Resource and E nergy Economics 24: 13

- 31. Horne, P., P.C. Boxall, and W.L . Adamowicz. 2005. “Multiple

- use management of forest recreation sites: a spatially explicit choice experiment.” Forest Ecology and Management 207(1/2): 189

- 99. Johannesson, M. 1997. “Some Furth er Experimental Results on Hy pothetical Versus Real Willingness to Pay .” Applied Economics Letters 4: 535

- 536. Johnston, R.J., E.T. Schultz, K. Segerson and E.Y. Besedin. 20 10

. “ Bioindicator

- Based Stated Preference Valuation for Aquatic Habitat and Ecosy st em Service Restoration ” , In Bennett, J. ed. International Handbook on Non

- Marketed Environmental Valuation

. Cheltenham, UK : Edward Elgar, forthcoming. Johnston, R.J. 2006. “I s Hy pothetical Bias Universal? Validating Contingent Valuation Responses Using a Binding Public Referendum.” Journal of Environmental Economics and Management 52(1):469

- 481. Johnston, R.J., and D.P. Joglekar. 2005. “Validating Hypothetical Survey s Using Binding Public Referenda: Implications for Stated Preference Valuation.” American A gricultural Economics Association (AAEA) Annual Meeting, Providence, July 24

- 27. Johnston, R.J., J.J. Opaluch, M.J. Mazzotta, and G. Magnusson. 2005. “Who Are Resource Non

users and What Can They Tell Us About Non

- use Values? Decomposing User and Non

user Willingness to Pay

for Coastal Wetland Restoration.” Water Resources Research 41(7), doi:10.1029/2004WR003766. Johnston, R.J., E.Y. Besedin, and R.F. Wardwell. 2003a. “Modeling Relationships Between Use and Non

- use Values for Surface Water Qualit y: A Meta

- Analy sis.” Water Resources Research 39(12): 1363. Johnston, R.J., S.K. Swallow, T.J. Ty rrell, and D.M. Bauer. 2003b. “Rural Amenity Values and Length of Residency .” American Journal of Agricultural Economics 85(4): 1000

- 1015. Johnston, R.J., G. M agnusson, M. Mazzotta, and J.J.Opaluch. 2002a. “Combining Economic and Ecological Indicators to Prioritize Salt Marsh Restoration Actions.” American Journal of Agricultural Economics 84(5): 1362

- 1370.Johnston, R.J., S.K. Swallow, C.W. Allen, and L .A. Smith. 2002b. “Designing Multidimensional

Environmental Programs: Assessing Tradeoffs and Substitution in Watershed Management Plans.” Water Resources Research 38(7): IV1

- 13. .Johnston, R.J., S.K Swallow and T.F. Weaver. 1999. “Estimating Willingness to Pay an d Resource Trade

- offs With Different Pay ment Mechanisms: An Evaluation of a Funding Guarantee for Watershed Management.” Journal of Environmental Economics and Management 38(1): 97

- 120. Johnston, R.J., T.F. Weaver, L.A. Smith, and S.K. Swallow. 1995. “Contingent Valuation Focus Groups: Insights From Ethnographic Interview Techniques.” Agricultural and Resource Economics Review 24(1): 56

- 69. Just, R.E., D.L . Hueth, and A. Schmitz. 2004. The Welfare Economics of Public Policy: A Practical Approach to Projec t and Policy Evaluation . Edward Elgar Publishing, Cheltenham, UK and Northampton, MA. Kaplowicz, M.D., F. Lupi, and J.P. Hoehn. 2004. “Multiple Methods for Developing and Evaluating a Stated

- Choice Questionnaire to Value Wetlands.” Chapter 24 in Methods for Testing and Evaluating Survey Questionnaires , eds. S. Presser, J.M. Rothget, M.P. Coupter, J.T. Lesser, E. Martin, J. Martin, and E. Singer. New York: John Wiley and Sons. Kobayashi, M.K. Rollins , and M.D.R Evans. 2010. “ Sensitivity of WTP Estimates to Definition of 'Yes': Reinterpreting Expressed Response Intensity

. ” Agricultural and Resource Economics Review 39(1): 37

- 55. Kuhfeld, W.F. 2009. “ Experimental Design: Efficiency , Coding and Choice Designs. ” SAS Institute. http://support.sas.com/techsup/tnote/tnote_stat.html#market

. Layt on, D.F. 2000. “Random coefficient models for stated preference survey s.” Journal of Environmental Economics and Management 40(1): 21

- 36. Li st, J.A. 2001. “Do Explicit Warnings Eliminate the Hy pothetical Bias in Elicitation Procedures? Evidence from Field Auctions for Sportscards.” American Economic Review 91(5): 1498

- 1507. Loomis, J., T. Brown, B. Lucero, and G. Peterson. 1996. “Improving Val idity Experiments of Contingent Valuation Methods: Results of Efforts to Reduce the Disparity of Hy pothetical and Actual Willingness to Pay .” Land Economics 72(4): 450

- 461.Louviere, J.J., D.A. Hensher, and J.D. Swait. 2000. Stated Preference Methods: Ana lysis and

Application . Cambridge, UK: Cambridge University Press. Maddala, G.S. 1983. “Limited

- Dependent and Qualitative Variables in Econometrics.” Econometric Society Monographs No. 3, Cambridge University Press, Cambridge. Mazzotta, M.J., J.J. Opaluch, G. Magnuson, and R.J. Johnston. 2002. “Setting Priorities for Coastal Wetland Restoration: A GI S

- Based Tool That Combines Expert Assessments And Public Values.” Earth System Monitor 12(3): 1

- 6. McConnell, K.E. 1990. “Models for Referendum Data: The Struc ture of Discrete Choice Models for Contingent Valuation.” Journal of Environmental Economics and Management 18(1): 19

- 34. McFadden, D., and K. Train. 2000. “Mixed Multinomial L ogit Models for Discrete Responses.” Journal of Applied Econometrics 15(5): 447

- 470. Mitchell, R.C., and R.T. Carson. 1981. An Experiment in Determining Willingness to Pay for National W

ater Quality Improvements . Preliminary draft of a report to the U.S. Environmental Protection Agency . Resources for the Future, Inc., Washington. Mitchell, R.C., and R.T. Carson. 1984. A Contingent Valuation Estimate of National Freshwater Benefits: Technical Report to the U.S. Environmental Protection Agency. Washington, DC: Resources for the Future. Mitchell, R.C., and R.T. Cars on. 1989. Using Surveys to Value Public Goods: The Contingent Valuation Method . Resources for the Future, Washington, D.C. Morrison, M., and J. Bennett. 2004. Valuing New South Wales rivers for use in benefit transfer. Australian Journal OfAgricultural And Resource Economics 48(4): 591

- 611. Murphy , J.J., T. Stevens, and D. Weatherhead. 2004. “Is Cheap Talk Effective at Eliminating Hy pothetical Bias in a Provision Point?” Working Paper No. 2003

- 2. Department of Resource Economics, University of Massachusetts, Amherst. Olsen, D., J. Richards, and R.D. Scott. 1991. “Existence and Sport Values for Doubling the Size of Columbia River Basin Salmon and Steelhead Runs.” Rivers 2(1): 44

- 56. Opaluch, J.J., T.A. Grigalunas, M. Maz zotta, R.J. Johnston, and J. Diamantedes. 1999. Recreational and Resource Economic Values for the Peconic Estuary. Prepared for the Peconic Estuary

Program. Peace Dale, RI: Economic Analy sis Inc. 124 pp.Opaluch, J.J., S.K. Swallow, T. Weaver, C. Wessel ls, and D. Wichelns. 1993. “Evaluating

impacts from noxious facilities: Including public preferences in current siting mechanisms.” Journal of Environmental Economics and Management 24(1): 41

- 59. Poe, G. L ., J.E. Clark, D. Rondeau, and W.D. Schulze. 2002. “Provision Point Mechanisms and Field Validity

Tests of Contingent Valuation.” Environmental and Resource Economics 23: 105

- 131. Poe, G.L ., M.P. Welsh, and P.A. Champ. 1997. “Measuring the Difference in Mean Willingness to Pay when Dichotomous Choic e Contingent Valuation Responses are not Independent.” Land Economics 73(2): 255

- 267. Powe, N.E. 2007. Redesigning Environmental Valuation: Mixing Methods within Stated Preference Techniques . Cheltenham, UK: Edward Edgar. Powe, N.A., and I.J. Bateman. 2004 . “Investigating Insensitivity to Scope: A Split

- Sample Test of Perceived Scheme Realism.” Land Economics 80(2): 258

- 271. Ready , R.C., P.A. Champ, and J.L . L awton. 2010. “Using Respondent Uncertainty to Mitigate Hy pothetical Bias in a Stated Choice Experi ment.” Land Economics 86(2): 363

- 381. Ready , R.C., J.C. Whitehead, and G.C. Blomquist. 1995. “Contingent Valuation When Respondents are Ambivalent.” Journal of Environmental Economics and Management 29(2): 181

- 196. Smith, V. K., and C. Mansfield. 1998. “Bu yi ng Time: Real and Hy pothetical Offers.” Journal of Environmental Economics and Management 36: 209

- 224. Schkade, D.A. and J.W. Pay ne. 1994. “How People Respond to Contingent Valuation Questions: A Verbal Protocol Anal ysis of Willingness to Pay for an Env ironmental Regulation.” Journal of Environmental Economics and Management 26: 88

- 109. Train, K. 1998. “Recreation Demand Models with Taste Differences Over People.” Land Economics 74(2): 230

- 239. U.S. Department of Labor, Bureau of L abor Statistics. 2009. Table 1: Civilian workers, by major occupational and industry group. September 2009. http://www.bls.gov/news.release/ecec.t01.htm

. U.S. EPA. 2000. Guidelines for Prepari ng Economic Analyses . (EPA 240

- R

- 00

- 003). U.S. EPA, Office of the Administrator, Washington, DC, September 2000.U.S. EPA. 2006. Peer Review Handbook 3rd Edition . (EPA 100

- B

- 06

- 002). U.S. EPA, Science

Policy Council, Washington, DC, 2006. Versar. 2006. Comments Summary Report: Peer Review Package for "Willingness to Pay Survey Instrument for §316(b) Phase III Cooling Water Intake Structures.” Prepared b y Versar Inc., Springfield, VA. Viscusi, W. Kip, Joel Huber, and Jason Bel l. 2008. “The Economic Value of Water Qualit y.” Environmental and Resource Economics 41(2): 169

- 187. Vossler, C.A., and M.F. Evans. 2009. “Bridging the gap between the field and the lab: Environmental goods, policy maker input, and consequentiality .” Jou rnal of Environmental Economics and Management 58(3):338

- 345. Vossler, C.A., and J. Kerkvliet. 2003. “A Criterion Validity Test of the Contingent Valuation Method: Comparing Hypothetical and Actual Voting Behavior for a Public Referendum.” Journal of Envi ronmental Economics and Management 45(3): 631

- 649. Whitehead, J.C., G.C. Blomquist, T.J.Hoban, and W.B. Clifford. 1995. “Assessing the Validity and Reliability

of Contingent Values: A Comparison of On Site Users, Off Site Users, and Non

- users.” Journal of Environmental Economics and Management 29(2): 238

- 251. Whitehead, J.C., and P.A. Groothuis. 1992. “Economic Benefits of Improved Water Quality: a Case Study

of North Carolina's Tar Pamlico River.” Rivers 3: 170

- 178.