PEER REVIEW REPORT 

Peer Review of Survey Instrument and Charge Questions for Chesapeake Bay 

Stated Preference Survey

I. Summary of EPA Responses to Peer Review 

II. Response from Dr. Kevin J. Boyle Professor, Department of Agricultural and Applied 

Economics, Virginia Polytechnic Institute and State University

III. Response from Dr. John C. Whitehead Professor and Chairperson, Department of Economics, 

Appalachian State University

IV. Response from Dr. Robert J. Johnston Director, George Perkins Marsh Institute and

Professor, Department of Economics, Clark University

Appendix

A.1 Peer Review Package prepared by Abt Associates, Inc. Sent 6/25/2012
Summary of EPA Response to Peer Review of Chesapeake Stated Preference Survey

The stated preference survey instrument was subjected to a letter review conducted by a contractor.

There were three peer reviewers with expertise in stated preference methods, with at least one reviewer 

having applied stated preference experience in a water quality or ecosystem services context to estimate 

non-use value. Consistent with peer review guidance, an alphabetized list of potential peer reviewers was 

provided to the contractor, although the contractor was not required to select from this list. The 

contractor reviewed this list and other qualified candidates and chose the peer reviewers for the survey 

instrument. The peer reviewers were Dr. Kevin J. Boyle Professor, Department of Agricultural and 

Applied Economics, Virginia Polytechnic Institute and State University; Dr. Robert J. Johnston Director, 

George Perkins Marsh Institute and Professor, Department of Economics, Clark University; and Dr. John 

C. Whitehead Professor and Chairperson, Department of Economics, Appalachian State University.

1. Administering the input version, endpoint version, or both

All peer reviewers advised that we administer only one version of the survey but differed as to which was 

preferred. Dr. Kevin Boyle and Dr. John Whitehead favored the input version in part because the 

endpoint version would not fully capture total values. In contrast, Dr. Robert Johnston made a convincing 

case for the endpoint version, citing economic theory and the difficulties focus group and interview 

subjects had in understanding the attributes in the input version (especially DO, see below).

Action taken 

Administer the endpoint version only

Dr. Johnston made a strong argument for using the endpoint version. Using a single wellresearched ecological production function is preferable to presenting levels of ecological inputs 

and allowing each respondent to impose their own subjective predictions as to how these inputs 

impact ecological endpoints that directly enter their utility functions. Although the endpoint 

version of the survey focuses only on a subset of critical components of the Bay, doing so will 

produce a more accurate, albeit conservative, estimate of the benefits from improving the quality 

of the Bay.

Furthermore, since the quantitative links between ecological inputs and endpoints are not 

imbedded within the endpoint version of the survey, if ecological modeling improves in the future 

(i.e. the quantitative links become more accurate), data collected with the endpoint version of the 

survey allow the flexibility to estimate benefits based on different forecasts of endpoint levels. In 

contrast, a well designed input version of the survey would explicitly include these links within 

the survey text, and so the survey responses depend directly on the quantitative links presented in 

the information section of the survey. 

Options discussed but not taken 

Administer two versions as planned

Given that all three reviewers independently advise against administering both versions and make 

some pretty good arguments in doing so, we decided to no longer consider this option. 

Administer the input version only

The upside of the input version is that it is more likely to capture total value because the input 

attributes are at a higher level in the ecological process, and therefore encompass a broader 

spectrum of Bay characteristics that individuals care about. However, given the drawbacks 

discussed, it remained clear that pursuing the endpoint version, and obtaining a more accurate yet 

conservative benefit estimate, was preferred. 
 

2. Whether to include the Dissolved Oxygen (DO) attribute or not

The DO attribute was not readily understood by respondents and they did not make economic decisions 

involving the attribute. Because DO requires respondents’ subjective ecological production function the 

input version will not produce meaningful welfare estimates. To some extent this is true for sub-aquatic 

vegetation (SAV) too. 

Action taken 

Drop the DO attribute and rely only on the endpoints version of the survey

The purpose of the DO attribute was to convey a quantitative measure of “overall Bay health.” 

Instead we chose to rely on endpoints that are more likely to enter a household’s utility function 

directly. Using endpoints is likely to provide an underestimate of total benefits but the resulting 

WTP responses will be more reliable. 

Options discussed but not taken

Rely on current version of the input survey with some improvements

Most of the cognitive difficulty with DO attribute seemed to be related to the description of goals 

and technical aspects rather than understanding what dissolved oxygen itself means. These 

difficulties could be addressed in the attribute description, but doing so does not address the issue 

of subjective ecological production functions.

Better define the dissolved oxygen attribute

DO might still be a useful attribute if we can characterize what meeting a DO goal means in terms 

of endpoints or ecological functioning. DO criteria are generally set at levels that “fully protect 

the survival, growth and propagation” of fish and shellfish. The Bay Report Card also 

characterizes meeting DO standards categorically, including letter grades and brief descriptions in 

terms of letter grades. But it’s not clear that this depicts the ecological production function 

enough to overcome the essential criticism. Similar concerns would still remain regarding the 

SAV attribute. 

3. The endpoint version of the survey could introduce omitted variables bias

The ecological changes from reducing nutrient and sediment loads to the Bay are too broad to capture 

with a small set of endpoint attributes. Reviewers are concerned that the marginal value for a given 

attribute will be overestimated because respondents may perceive that attribute as being positively 

correlated with an omitted variable they care about. At the same time, an endpoint version is likely to 

underestimate total economic value because not all beneficial changes will be captured in the choice 

scenarios.

Action taken

Add striped bass to endpoint version of the survey and add text to encourage respondents to only 

consider attributes represented in the choice scenarios

Peer reviewers and focus groups identified the following attributes that people care about and 

may be affected by policy, but are not included on the endpoint version of the survey: birds, 

striped bass/game fish, and rivers and streams. However, adding another three attributes without 

making any other changes would place undue cognitive burden on the respondents. Since striped 

bass are the most popular game fish in the Bay and we have the ability to model population 

changes, we have decided to add striped bass population to the set of attributes on the endpoint 

version of the survey. We will also add text reminding people that only the changes shown in the 
choice scenarios should be considered when voting, and that all other factors and attributes do not 

vary across alternative scenarios.

Options discussed but not taken

Rely on current version of the endpoint survey

Some of the concern over omitted variables bias could be mitigated by making editorial changes 

to the survey but leaving the set of endpoint attributes unchanged. Such changes would include 

choosing a different picture for the cover and stressing that attributes that are not included in the 

choice question would not be affected by the “additional pollution reduction programs” posed in 

the survey. This would not address the underestimation of total economic value, but our estimates

could be characterized as conservative since they only capture the improvements explicitly 

described on the survey. However, the exclusion of striped bass may lead to a significant 

underestimate, since it was deemed important by focus group respondents.

Include a row at the top of the choice question listing attributes that would not change from 

baseline under either policy. This could be visually displeasing and redundant, and so instead this 

text will be added and emphasized in pages leading up to the choice questions.

Add multiple attributes to endpoint version and make additional changes to reduce cognitive 

burden. 

Some of the changes we discussed were

Switch to a two alternative design (No Action and one Policy option)

Combine crabs and oysters into one “shellfish” category (but this poses two difficulties: 

(i) would have to rely on percent changes because units are different, and (ii) establishing 

a reference level becomes difficult)

Only vary 3 attributes at a time but allow the set of varying attributes to change from 

question to question

4. The lakes attribute could be problematic 

Dr. Boyle identified two main reasons the lakes attribute could be problematic. First, it is a compound 

attribute which is not standard practice and prevents us from finding the marginal benefit of any single 

sub-attribute. Second, people outside the watershed could be thinking about local lakes and voting for a 

program based on benefits that would not accrue to them. 

Action taken

Change the lakes metric in the choice questions, tell people whether they live in the watershed or 

not, and further emphasize that lakes outside the watershed are unaffected by the additional 

pollution reduction programs posed in the survey. 

By redefining the lakes attribute and making some editorial changes to the survey we believe we 

can address the major concerns regarding this attribute. We will stress that changes would only 

occur within the watershed and use the introduction letters and survey cover letters to tell 

respondents whether they live inside or outside of the watershed. We will reduce the set of lake 

categories to “high algae” and “low or moderate algae” to allow the complete characterization of 

lakes in the watershed with a single metric (i.e. the percent of lakes in the “low or moderate” 

category.)

Options discussed but not taken

Drop lakes attribute from surveys sent to non-watershed states.

This would prevent people outside the watershed from mistakenly considering local benefits and 

shorten the survey for that sample stratum. It would not allow us to capture use or nonuse value 
that people outside the watershed hold for lakes in the watershed. Furthermore, reducing the 

number of attributes provided to one geographic strata of the population could confound 

comparison of results, response rates, etc. across respondents in and outside of the watershed.

Split up the lake sub-attributes but don’t vary them independently.

This would raise the number of attributes to 7 (including cost) and that’s before we add striped 

bass. If we keep the same relationship between the lake sub-attributes it would not affect our 

sample size requirements, but could pose additional cognitive burden on respondents.

Administer two different surveys, separating the impacts to lakes/rivers/streams and the Bay

mainstem and tidal waters

We are too resource constrained (both in time and funding) to pursue this option but it was 

discussed. 

5. Goal reference levels are arbitrary 

Using policy goals as reference points for the attributes might encourage symbolic responses about ‘doing 

enough’ for the environment. Further, responses could also be subject to manipulation because policy 

goals are endogenous, subjective, and change over time. An unambiguous and objective reference level 

for each attribute (such as a recent historical reference) would not be subject to such bias.

Action taken 

Use current and recent historical levels as a reference point

Focus groups showed that “pristine conditions” or reference levels from the distant past were not 

meaningful to respondents. Instead we will choose a recent point in time for which data are 

available on all attributes to use as a reference level. This allows respondents link these 

quantitative attribute levels with recent their recent experiences with the Bay and watershed lakes, 

thus making the environmental commodity more tangible. 

Option discussed but not taken

Drop reference levels 

Early versions of the survey did not include reference levels, but focus group participants wanted 

some way to frame the current and projected attribute levels.
II. Response from Dr. Kevin J. Boyle Professor, Department of Agricultural and 

Applied Economics, Virginia Polytechnic Institute and State University

Comments on:

Peer Review Package for Stated Preference Estimation of Benefits from the

Chesapeake Bay TMDLs

July 16, 2012

Overall impression:

The survey instrument appears to be carefully designed using appropriate procedures.

A choice experiment is a common stated preference method in the nonmarket valuation 

literature. I believe there are a number of changes that could be made to improve the survey

instrument. My comments do not imply that this is a poorly designed instrument. Rather, the

careful work done on the instrument to date allowed me to provide what I hope are helpful 

comments for refining the survey instrument to enhance the usefulness of value estimates for

[policy analyses.

Detailed comments are listed after I answer the charge questions. Please see these comments

to support ideas I propose in responses to the charge questions.

Please feel free to contact me if you have questions.

General Impression of Survey Instrument and Proposed Study

1. Extent to which the survey results are likely to provide welfare estimates of improving water

quality in the Chesapeake Bay. Specifically,

a. the ability of the survey to provide total values

I believe the survey is capable of measuring total values, but do have some concerns.

The survey starts with reference primarily to recreation and does not set the tone for

respondents to think larger in terms of total values. I am concerned that the inconsistent

references to lakes and streams in the survey instrument may result in some respondents,

especially those outside the watershed, valuing local water quality changes outside of

the watershed.

b. appropriateness of the proposed choice experiment survey for the estimation of marginal

willingness to pay for individual attributes

The survey seems well designed to estimate marginal values for the attributes included

in the design, but I have an important concern. I wonder if there will be an omitted

variable bias from people thinking of other attributes, e.g., birds given the photograph

on the cover. The survey opens the interconnectedness of aquatic ecosystems and I
wonder if omitting attributes that people think about, but cannot be modeled, will lead

to omitted variable biases in estimates of attribute parameters. Are respondents using

the “iconic” attributes to signal larger changes in the aquatic ecosystems and are

different people making different assumptions? Are some people valuing more than

you intend and are others valuing the wrong changes? This is not simple heterogeneity

of preferences, but a fundamental issue of the definition of the change in quality

valued. You can include attributes in the design that cannot be modeled, but 

respondents care about. These attributes that people care about, but cannot be modeled,

can be held constant for conservative value estimates for policy. This would help avoid

omitted variable bias in estimates of preference parameters, which could confound

value estimation for policy analyses.

c. appropriateness of a choice experiment survey format for the elicitation of 

preferences in this case (i.e., compared to alternatives such as contingent 

valuation).

A choice experiment is convenient to the analyst and decision makers, but may not

effectively convey the interconnectedness of the ecosystem. A contingent- valuation

survey could do this without the implied assumption to respondents of additive

separability of attributes. The limitation to specific attributes may lead respondents to

bring other attributes into consideration given the ecosystem effects, using the specified

attributes as indicators of general ecosystem effects. I am not saying CV survey should

be conducted, but am noting this is a worthy

discussion. I think the relative trade-offs of these two stated-preference methods, for this 

particular application, should be discussed in the final report. May also want to add

some more directed follow-up questions after the choice questions to make sure this is 

not a problem in how respondents answer the choice

questions.

2. Overall quality and design of the proposed survey. 

The overall quality and design of the survey is very good. I do see a number of ways to

tighten the survey design and a couple of major issues. Examples include:

• I would drop the endpoint design. 

• Wildlife needs to be defined.

• References to lakes and streams need to be more consistent.

Survey Instrument

3. Adequacy of background information to enable the respondents to answer the choice 

questions. Is the information necessary, sufficient, and accurate?

I believe there are a number of needed improvements:

• Consistent reference to lakes and streams in the watershed so that respondents will not

value water quality outside the watershed. 
• Definition of wildlife.

• Clear role for streams; regularly mentioned, but quietly disappear when one gets to the

choice questions.

4. Does the survey provide appropriate and sufficient introductory and debriefing questions to, for

example, identify protest responses and symbolic responses, test retention and

understanding of information, elicit non-use values, etc.? Are there any additional 

introductory or debriefing questions you feel may be useful for the analysis?

• I believe the introduction is too focused on use for a total value survey and encourage

adding at least one question that gets respondents thinking about total values.

• Debrief questions do not address if respondents are valuing changes outside the watershed

or are correctly interpreting attributes and levels. I think two questions need to be added

to the debriefing section.

5. Does the survey place sufficient and appropriate emphasis on the budget constraint facing

respondents, and on potential substitute uses of their budget?

Yes, but I have a concern about the overuse of the word “cost” in this section of the survey. 

I wonder if all of the references to cost here will encourage people to think about the cost of

the program (cost calculators) and compare bid times households to see if it is reasonable,

not think about their household values.

6. Considering all four versions of the survey, are the program options under consideration

appropriately and sufficiently characterized? For example, given the focus group

summary provided, are the appropriate attributes included in the choice experiment?

• I worry that omitting attributes that are of concern to respondents, but cannot be 

modeled physically, will lead to omitted variable biases in the estimation of implicit

prices. The likely outcome is over estimation.

• Are respondents using the “iconic” attributes to signal larger changes in the aquatic 

ecosystems and are different people making different assumptions? Are some people

valuing more than you intend and are others valuing the wrong changes?

• Will people be looking for birds (picture on survey cover) or striped bass, and make

assumptions on the absence?

• Scientific modeling limitations are not necessarily reasons to omit what may be 

important attributes to respondents. You can include these attributes in the design, but

hold their levels constant for conservative value estimates for policy. This would help

avoid omitted variable bias in estimates of preference parameters, which could

confound value estimation for policy analyses.

• I believe the Lakes attribute is confounded due to the agglomeration of sub attributes 

within this attribute. Lake condition concerns me because it is a compound attribute

and you cannot be sure what piece of information people are reacting to – color, clarity

or fish. Nor do we know what they are assuming about fish stocks and catchability. I

wonder if respondents will implicitly think that bass will increase by 33% with

Program A because of the way you have linked clarity and fish in the table and text

preceding, or 67% if you add percent to visibility in this box. What if you are not a
resident in a state in the watershed, state-specific information is not relevant? I wonder

if levels and feet of clarity, two numbers, will confuse respondents and which number

they are responding to and if everyone is responding the same? I do not think the

estimated implicit price for this attribute will be useful for estimating ancillary,

upstream benefits from the Bay.

7. Regarding the “Input” version of the survey, are the links between attributes in the program

options (such as Dissolved Oxygen) and aspects of the Bay respondents may directly 

care about appropriately described? Are there ways to improve the description of those

links?

Employing the input and endpoint versions may provide unnecessary complexities in policy 

analyses. If they provide the same value estimates this could be construed as a failure of

scope and undermines the survey. If they provide different value estimates and are valuing

the same ecological conditions, just different attribute metrics, then this could be viewed as a

violation of procedural invariance. In this later case, which values would you propose for

policy? This seems to be a major issue left undecided at the design table. I recommend

deleting the endpoint versions and reallocating the sample to the input versions because you

are working with minimal sample sizes.

8. Does the survey incorporate appropriate language to ameliorate symbolic responses and 'yea

saying'?

• I do not think the survey has done enough to address respondents interpreting attributes

as symbols of larger ecosystem effects than intended.

• I suggest adding cheap talk script to reinforce people from yea saying.

9. Does the survey take appropriate steps to minimize hypothetical and strategic bias, including

sufficient steps to emphasize consequentiality?

I have made numerous suggestions in my detailed comments to enhance 

consequentiality. I think the wording, as currently presented, is too equivocal to 

reinforce consequentiality.

10. Is the format of the choice questions appropriate according to current standards in the stated

preference literature (e.g., allowing for a status quo option, encouraging neoclassical

tradeoffs, order and presentation of attributes, etc.)

Yes, with a few of exceptions:

• Repetition of status quo conditions in the first two columns may be confusing. 

• Lakes is a compound attribute with three sub attributes, which is not standard 

practice.

• Cost is not clearly in an attribute row, but is in the response row,

• Carrying cost forward into perpetuity is not common. It is more common to do a oneshot payment of payment for a fixed period of time.
11. Is there evidence of fundamental or critical flaws in the survey? Such as bias in the survey 

or presented information?

Three key concerns stand out to me:

• Lakes are not clearly defined as within the watershed.

• Streams are introduced, but not adequately addressed in the choice questions. There is a

one-bullet dismissal within the lake description.

• The lake attribute is confounded and I doubt the implicit price estimate will be useful

for policy analyses.

Study Design

12. Are the proposed attribute levels and use of a constant versus declining baseline 

appropriate for estimating the benefits of the TMDL?

In Section 1.1, 1st para., you state “These TMDLs called for reductions of 25, 24, and 20%,

respectively, of these pollutants (EPA 2011)”. The attribute levels do not seem refined enough

to distinguish between these different levels. The attribute levels actually allow estimation of

nonmarginal changes, not marginal in the true sense.

I like the idea of investigating two different baselines. Too often the assumption is a 

constant baseline when this is not in fact the case. This comparison is important for the

current policy debates and to inform the design of future studies.

13. Is the sampling plan and proposed geographic stratification appropriate for estimating the

total benefits of the TMDL, including nonuse values?

Detailed Comments – Developed independently and some may be contradictory based on

flow of the survey information. Some are repeated in answers to charge questions above.

P1-1 – Blue Ribbon Panel (2006) report is not in the references.

P 1-1, 3rd para. – Implicitly implies only use values for the first group, which is likely a 

false assumption. Thus, hedonic and RUM models will provide lower bounds of 

benefits.

In addition, these stated preference models are challenged for nonmarginal changes such

as proposed for the Chesapeake in terms of observed vs. unobserved levels of water

quality, marginal vs. nonmarginal changes in quality, and shifts in location or usage that

might occur.

P 1-1, 4thpara. – Meeting TMDLs will require land-use changes, which also may 

generate benefits that are not being captured in the described analyses.
P 1-2, 1st para. – Asserts, implicitly, a choice experiment (CE), but no discussion of whether a

contingent-valuation (CV) study may have been more appropriate. A CE is convenient to

the analyst and decision makers, but may not effectively convey the interconnectedness of

the ecosystem. A CV survey could do this without the implied assumption to respondents of

additive separability of attributes. I am not saying CV survey should be conducted, but am

noting this is a worthy discussion.

P 1-2, 2nd para. – People do not care about grasses directly. In some cases grasses are a

negative and others positive to recreational experiences. This may carryover over to

nonuse values as well.

P 1-2, 2nd para. – Employing the input and endpoint versions may provide unnecessary 

complexities in policy analyses. If they provide the same value estimates this could be

construed as a failure of scope and undermines the survey. If they provide different value

estimates and are valuing the same ecological conditions, just different attribute metrics,

then this could be viewed as a violation of procedural invariance. In this later case, which

values would you propose for policy? This seems to be a major issue left undecided at the

design table.

P 1-2, last para. – Are respondents using the “iconic” attributes to signal larger changes in the

aquatic ecosystems and are different people making different assumptions? Are some people

valuing more than you intend and are others valuing the wrong changes? This is not simple

heterogeneity of preferences, but a fundamental issue of the definition of the change in

quality valued.

P 1-3, top – This is a continuation of the concern from the bottom of the previous page.

Will people be looking for striped bass and make assumptions on the absence? You have

clarity and DC to address not using the index, but do not seem to have a way to directly 

address these other attributes. Scientific modeling is not necessarily a reason to omit what

may be important attributes to respondents. You can include these attributes in the design,

but hold their levels constant for conservative value estimates for policy. This would help

avoid omitted variable bias in estimates of preference parameters, which could confound

value estimation for policy analyses.

P 1-3, 7thpara. – What do respondents assume about where changes will occur and what do

you want them to assume?

P 1-4, top – You reference the fourth focus group here. It would be useful to document the

number of focus groups and the objectives of each group.

P 1-5, Section 1.2.6 – CV can be used to estimate these values; it just requires multiple scenarios 

and larger sample sizes.

P 1-5, Section 1.2.7 – I like the idea of investigating two different baselines. Too often the

assumption is a constant baseline when this is not in fact the case. This comparison is

important for the current policy debates and to inform the design of future studies.
P 1-6, 2nd full para. – Ngene could be used to develop a more efficient attribute design. 

P 1-6, 3rd full para. – I recommend, given this information, just implementing the input 

version. See my comments above (P 1-2, 2nd para.).

Table 1 – Linking color and clarity does not match my first –hand knowledge from research in

New England. Green water is due to algae blooms from nutrient enrichment. Brown is 

typically due to natural tannins in the water. Clear lakes are often a greenish color, not blue. 

Trout are in some greenish waters; this is the case for the lake where my family has a camp,

and rough fish occur in some very clear lakes. I was just on a number of lakes in the

northeast for other research last week and the color scheme here does not match what I

observed. I am concerned the color/fish categories, linked to clarity, will confuse some

respondents and wonder if color is actually needed and fish can be placed as a separate

attribute?

Table 3 – In Section 1.1, 1st para. you state “These TMDLs called for reductions of 25, 24, and

20%, respectively, of these pollutants (EPA 2011)”. The attribute levels do not seem refined

enough to distinguish between different levels with differences of 1 and 4%.

Table 5 – It will be important to state that you cannot generalize to specific states with these

sample sizes. Is this going to be a problem, particularly in the Bay States, where there may 

be a demand for state-specific benefit estimates?

Given these sample sizes, which are modest, I emphasize my recommendation to remove

the endpoint cells so you can focus all observation on the primary policy valuations.

Survey Comments Based on Attachment A

Survey cover – You symbolically imply that birds are an attribute with the cover picture.

While this is very attractive, I strongly recommend that this photograph be replace with a

more neutral picture; perhaps a map of the watershed.

I would remove “Stated Preference” from the cover. This is techno talk; remember you are

writing for about an 8thgrade audience and should avoid academic jargon.

Survey, 1st page, 1st para. – To enhance consequentiality “The survey’s purpose is to collect

information to help policymakers will used to make decisions that affect water quality,

wildlife and recreation.”

Survey, 1st page, 2nd para. – First sentence seems to be abrupt.  “We will present a number of

plans and ask want to know which of these plans you would vote for if a referendum were 

held today.”

Survey, 1st page, 3rd para. – Not really giving them an opportunity to “voice your opinions”,

rather indicate their preferences for likely alternative plans. 
 Survey, 1st page, 4th

para. – “agencies” instead of “offices”

Survey, 2nd page – Is there potential confusion showing the watershed in green when green

is linked to water quality?

Survey, 2nd page – Does the “Freshwater lakes and rivers” box and arrow convey that these

occur throughout the watershed. For “Tidal Waters” you denote the bay with the crosshatched box, but do not describe the dark blue.

Survey, 2nd page, 1st para. – To clarify  “Water draining from these lands in the watershed

enters these waterways and eventually the Chesapeake Bay.”

Survey, 2nd page, 2nd para. – Sentence leaves reader hanging and begs for explanation of the

points raised.

Survey, 3rd page and elsewhere – Answer categories are sometimes check boxes, sometimes 

numerical and other times alphanumeric (e.g., DK). I have learned that one response format

should be consistently used to minimize respondent confusion.

I may have missed it, but I did not see the survey implementation mode implemented. I am

assuming that it is going to be a mail survey. If you are going to scan the returned surveys 

then check boxes are appropriate. If you are going to do manual data entry, then numbers 

reduce data entry errors. Alphanumeric do not have a role.

I believe that well-designed surveys have directions on how to answer all questions. For Q1

this might be “Please check all that apply in each column.” You have the direction for Q2.

Survey, 2nd and 3rd pages – Focus in boxes on 2nd page is recreation and you start with a

recreation question. I wonder if this is the best approach to get respondents to think in a

“total value” framework.

Q1 – Question is not consistent with answer box. You state” any freshwater lakes and rivers” in

the question, but the heading in the box says “in the watershed”. I think the later is what

you want. Tidal waters on 2nd page is not defined to include the dark blue areas, but this 

where most of these activities would occur for the Chesapeake.

Will people, especially in the bay states, struggle with where tidal waters end? I have had

this problem is coastal surveys of salmon anglers and needed to clearly define.

First answer category seems odd; swimming does not seem to go with other two, and

other two are variants of boating. Would respondents check water skiing and boating and

the same for jet skiing?

Q2 – Same first three comments as for Q1.

Need to make clear that these activities refer to those listed in Q1.
Is “five years” too long of a period, especially for local waters? Recall bias will be very 

problematic.

What about people who live on the water or drive by daily on way to and from work, or

work on or by the water. How will they answer this question? Not really a visit, but

interact on a daily basis.

Survey, 5thpage, 1st para. – Wildlife is use here and earlier without definition. Does it 

include fish? What about aquatic vegetation?

Survey, 5thpage, 2nd & 3rd para. – “Sediment” mentions rivers and streams, but does not 

clearly convey that this is carried to the Chesapeake. What about lakes? No specific waters 

mentioned for “Nutrients”. The information presented here needs to be more parallel and

encompassing of the watershed and links to the Chesapeake.

Survey, 6thpage, Practices ... – “till” is jargon; I suggest changing to “manage”. What is meant

by “better equipment”; I think you can be clearer.

Survey, 6thpage, Water ... – First statement is unnecessarily awkward; “Water will 

become clearer because ...” “... algae blooms because of less nutrients”

Survey, 6thpage, Ecosystem ... – Note, there is no link to birds so this supports removing bird

picture from cover. How frequently were birds mentioned in the focus groups? Is this a

potential source of omitted variable bias if you do not have a bird attribute?

Q5 – Emphasize (?) “closest” by underlining or something as this is a change in geographic focus.

I have always been taught that once you ask people to focus down in a survey you should not

go back to a larger scale, but after this focusing, which may not be in the Chesapeake, you do

bring respondents back to the Chesapeake watershed for the choice questions. Does this 

cause a point of confusion so that respondents outside the Chesapeake will value local lakes 

that are outside the watershed? This is one of my larger concerns.

Is 10 years too long given the time that people typically reside in the homes? Census 

data (1998) was 8.2 years for owner occupied and 2.1 years for renter.

Q6 – You are back to all “lakes and rivers”. This inconsistency needs to be cleaned up

as I fear it will be confusing respondents.

Survey, 8thpage – You have a bullet talking about different DC goals, but no similar bullet for

water clarity and grass habitat. Are they the same throughout the Bay? If yes, I think you

should tell respondents this.

Q7 – Refers to “these environmental goals” but no specific goals mentioned. Crossing out “any

of these” will make question more consistent with text.
You have been talking about lakes and rivers in the watershed, but are silent on them here.

What are the upstream consequences? I think these linkages should be developed here.

Survey, 9thpage, 1st para. – You mention “health” here and earlier, but do not define it.

I think it would be better to stick with the terminology you have been using (quality). You

do not have any threshold for healthy in your design.

Survey, 9th page, DO – Remove “Some plans being considered would achieve 100% in

2025.” You have said “no further action” above. Why add “will” to last column? Why 

not state “no change” like you do for clarity and grasses?

I think you need to state that current programs can prevent further declines, but will not

improve quality and this is why additional programs are needed.

Survey, 10thpage – You jump to lakes, but do not tell respondents if there are goals for lakes or

if this is just an additional benefit of meeting the Chesapeake goals. This is stated on page

11, but I think you need to change the order and place the statements here.

I have already commented on this table above.

Q8 – Again, why refer to lakes “closest to your home” when choice questions will only refer to

the Chesapeake watershed?

Survey, 11thpage, 1st bullet – “in” instead “of the watershed”.

Survey, 11thpage, 2nd bullet – Streams near poorly managed nutrient sources (e.g., large

dairies) can be very eutrophic. Seems odd to throw in one sentence on streams in the

middle of a section title “Lakes”.

Survey, 11thpage, text below 3rd bullet – For some respondents there will not be “part of the

watershed in your state”. And paragraph below, what are you going to do for “<state>” if the

watershed is not in a respondents state?

Survey, 11thpage, table – Why not state “no change” like you did for the bay? Add statement

again that current programs are sufficient to maintain ...

Survey, 12thpage. 1st para. – Change “but” to “and” in the last line.

Survey, 12th page. box – I recommend saying something about improvements in 

upstream lakes and streams and not just the negative statement here.

Survey, 13thpage – I think that the general cost of living is appropriate and the best payment

vehicle. I just worry about allowing it to continue forever. On the one hand, the perpetuity

may induce people to vote no, but on the other hand aggregation might result in an
unrealistically large total benefit estimate. There is no agreement on this in the literature, but

you could say that when the program will be paid for in 10 or 20 years.

Survey, 14thpage – Make Programs A and B separate, and then add a concluding 

sentence after the bullets explaining how they differ.

Survey, 15th page – Do you really need to repeat current conditions in first two columns.

I suggest making the first column title “Attributes” and removing current conditions, and add

“Maintain Current Conditions” after No Further Action in the second column.

If you are going to use arrows, then use the everywhere there is an increase and always say 

“no change” when there is no change.

It seems a little odd to have percentages for the Bay attributes, but not for lakes. 

While you have continually discussed streams above, they are dropped here. I think some

people will assume they are in lakes, others will not, and others will just be confused.

Red Directions are a concern. I have been taught to avoid red whenever possible in surveys 

because so many people are trained that red indicates an error of prohibited action. I suggest

changing the color of these directions and arrows.

Survey, 15thpage, DO, 1st two columns – Change “will meet” to “meets” so active voice like

the other bullets bellow. Add no change under No Further action like two attributed directly

below.

Survey, 15thpage, Water Clarity, 4thcolumn – “no change” not 67% increase.

Survey, 15thpage, Grasses, 4thcolumn – 50% increase over Program A, but should be

88% increase over current condition of 80,000.

Note there are problems like the two noted above in the three choice questions. 

 Survey, 15thpage. Lake Condition – Lake condition really concerns me because it is a 

compound attribute and you cannot be sure what piece of information people are reacting to

– color, clarity or fish. Nor do we know what they are assuming about fish stocks and

catchability. I wonder if respondents will implicitly think that bass will increase by 33%

with Program A because of the way you have linked clarity and fish in the table and text

preceding, or 67% if you add percent to visibility in this box. 

What if you are not a resident in a state in the watershed, state-specific information is not

relevant? 

No mention of fish under current conditions. 
I wonder if levels and feet of clarity, two numbers, will confuse respondents and which

number they are responding to and if everyone is responding the same? 

I do not think the estimated implicit price for this attribute will be useful for estimating 

ancillary, upstream benefits from the Bay.

Survey, 15thpage – To increase consequentiality, I suggest saying – “Please assume that you are

actually voting on these programs right now. State and federal decision makers will use the

outcome of the votes by yourself and others to choose the best program to improve water

quality in the Chesapeake Bay and its watershed.” Or something like this. This can replace

the last sentence in the first paragraph. I think you are using to indecisive wording to imply 

consequentiality.

I wonder if all of the references to cost here will encourage people to think about the cost of

the program (cost calculators) and compare bid times households to see if it is reasonable, not

think about their household values.

In the last paragraph, add at the end of the last sentence – “when answering each of the next

three sentences. 

I am not a big fan of cheap talk, but I think you will be criticized for not including cheap

talk. Since choice experiments generally give larger values than contingent valuation, I think

it is worthwhile to add whatever you can to make values conservative.

Q10, Q11 & Q12 – To replace 1st sentence to enhance consequentiality – “Please vote on the

three alternatives below.”

I suggest changing “per year” to “every year” for Programs A and B to reinforce to

perpetuity of the payment. Same with month.

Is there a reason why cost is not a separate row from the responses? This format could

result in too much or too little focus on Program cost and I do not know what outcome is 

more likely.

It is not clear whether the “cost of living in your household” is to you or your entire 

household.

Survey, 18thpage – I wonder if it would be better to replace the last bullet with – “Only one

program will be selected by decision makers.” I have not seen adding up as a problem in

choice questions and worry that the adding-up statement may create a new problem with

those who may not have thought about this before.

Q13 – You say “Chesapeake Bay tidal waters” here, but the label in the choice questions was

Chesapeake Bay and tidal waters”. 1, 4 and 7 in Likert scale are not aligned with labels. 

Think the direction should be – “Please circle one number” – so respondents will not circle
one of the three categories.

Q14 – This question is problematic because it may have varied from question to question. For

example, if an attribute had no change in one question it may have been not important, but

important in another question where there was an increase. I would not know how to

interpret the response to this question. I suggest two new questions:

• Please indicate if you never considered each of the following when choosing between

programs.

• Please indicate if you always considered each of the following when choosing between

programs.

The response directions could be – “Circle all that apply”.

Q15 – I see no reason for the prefix “Aside from costs”. This is only one attribute.

“Any lakes” or just “Lakes in the Bay watershed”?

I think there should be a “None of the above” category after the write-in category.

I am assuming this is an attempt to identify nonuse motivations, but the categories do not

seem to fully capture ecological motivations.

Q16 – Why not just “if the economy were better”?

3rd item – replace “contribute” with “pay”. Did not ask for a contribution and potentially 

undermines consequentiality.

4th item – Make specific that referring to Bay improvement programs.

Too weighted to reasons to vote for the status quo and need to be balanced with 

affirmative reasons for voting for a program.

Q17 – Remove the “skip” sentence because you have a neutral category, and may want to add a

don’t know answer category.

Only one “no” that is not a protest, need more balance of the answer categories here to.

Q18 – Remove the “skip” sentence because you have a neutral category, and may want to add a

don’t know answer category.

None of categories are invalid yes votes, and need more balance of the answer 

categories here to.

Q19 on – Make the same as census questions where ever possible. 

Q24 – Missing technical school/2-year degrees beyond high school.
Q26 – What about an environmental field related to the improvements?
III. Response from Dr. John C. Whitehead Professor and Chairperson, 

Department of Economics, Appalachian State University

Review of “Stated Preference Estimation of Benefits from the Chesapeake Bay TMDLs”

1. Extent to which the survey results are likely to provide welfare estimates of improving water 

quality in the Chesapeake Bay. Specifically,

a. the ability of the survey to provide total values

The input survey will provide total economic values since both users and nonusers are being 

surveyed and all endpoints are likely to be considered. I doubt if the endpoint survey will capture 

all use and nonuse values since the endpoints described in the survey are limited to two iconic 

shellfish species. Focus group participants mentioned game fish species (e.g., rockfish) and this 

seems like an obvious omission that will raise questions and/or lead respondents to think that 

there will be no impacts. This is especially true since game fish are mentioned on page 6 of the 

surveys and will be valued in the input versions.

My expectation is that the total value from the input survey will be greater than the total value 

from the endpoint survey due to these missing outcomes of the survey. The review document 

states that the endpoint versions are included for methodological reasons and they won’t 

contribute to the policy analysis since endpoints can be modeled. My suggestion is to drop the 

endpoint version unless there is an explicit hypothesis that can be tested between inputs and 

endpoints. Including both just to see what happens does not seem like a good enough reason.

The delineation of use and nonuse values may be compromised since only recreation experience 

is measured and not future recreation. Since water quality is expected to improve over the next

13 years, recreation participation is also likely to increase. Question #2 should be followed by a 

stated preference recreation participation question. This stated preference question could be used 

to separate future users from future nonusers to estimate pure nonuse values from the nonusers. 

The future users willingness to pay will contain both use and nonuse value.

b. appropriateness of the proposed choice experiment survey for the estimation of marginal 

willingness to pay for individual attributes

The choice experiment will effectively estimate marginal values for the inputs, although I don’t 

think that will be useful for the policy purposes of the results. As described in the review 

document, the choice experiment approach is chosen since the water quality modeling is 

uncertain and various combinations of the inputs may be used to developed different measures of 

total value for policy analysis. This is a legitimate reason.

With the endpoints, I am less certain that marginal values will be valid. My guess is that 

gamefish will be considered along with the shellfish species and these marginal values will be 

either overvalued or the total value from the will be underestimated.

c. appropriateness of a choice experiment survey format for the elicitation of preferences in this 

case (i.e., compared to alternatives such as contingent valuation).
The review document overstates the limitations of contingent valuation with respect to valuing 

different scenarios since multiple scenarios can be presented to respondents and these can be 

used to value different programs. However, I agree that the choice experiment approach is more 

flexible in this respect and likely to capture these different modeling outcomes.

There are no other revealed or stated preference approaches that are appropriate to the task of 

estimating use and nonuse values.

2. Overall quality and design of the proposed survey.

The survey has the potential to be of exceptionally high quality. However, it needs to be 

considerably tightened up after several rounds of proofreading.

For example (Version C):

On page 5 (“water quality in the bay”) the first sentence mentions water quality, wildlife habitat 

and recreational opportunities. Question #3 asks only about awareness of wildlife habitats. The 

awareness question should ask about water quality only or include water quality and recreational 

opportunities along with wildlife habitats. Language should be parallel and repeated.

The survey mentions pollution reduction programs but is vague on what these are. I would 

change the heading on page 6 to Pollution Reduction Programs. Also, the boxed information 

mentions water quality, wildlife habitat and recreation opportunities (repeating previous phrases 

which I strongly recommend) but the last heading introduces “ecosystem” for the first time.

Page 7 and throughout: Is there a reason why the “disagree” and “agree” answers are not 

included, other than formatting? It seems that answers 2 and 4 should be explicitly defined.

Page 8 (“environmental quality in the bay”) is confusing. The purpose seems to be defining the 

goals of the programs but measurements of these goals are emphasized. On the next page, two of 

the three inputs have goals mentioned subtly while the goal for water quality is ignored. It seems 

like page 8 would be a useful place to define measurement of the inputs and their goals (and 

delete things that are neither). For example:

Dissolved Oxygen

 Keep the first bullet point (Second bullet point: why do different DO goals for 

different areas matter?)

 Some plans being considered would achieve 100% of waters meeting DO goals in

2025

Water Clarity

 Keep the first two bullet points (third bullet point: why should a suggestion about 

water clarity preferences be included here?)

Describe the water clarity goal
Aquatic Grasses (delete “amount of” and “habitat”)

Keep the first bullet point

Scientists recommended grasses be restored to historic levels)

Page 8: The goals should be about water quality, wildlife habitat and recreation opportunities, 

not “environmental quality and aquatic life.” Again, new terms are being introduced and this 

confuses survey respondents.

Page 9: Repeat the goals from page 8 under the same headings. A down arrow should be 

included for DO along with % decrease for consistency. Respondents need a question after this 

information to allow them to gather their thoughts.

Page 11: The table requires some unnecessary thought. Presentation may be easier if levels 2 and

3 are presented in rows as on page 10 with current and future conditions in an additional column. 

Respondents need a question after this information to allow them to gather their thoughts.

Page 11: What is the goal for lakes?

Page 12: The information on this page is very confusing:

The heading is pollution reduction programs but programs are not described.

Goals are mentioned in the first sentence but these have not been clearly stated (see 

comment above).

 Goals are mentioned for water quality and wildlife but the inputs for which goals should 

be made are DO, water clarity and acres of bay grass.

 The information in the box does not seem to have a clear purpose.

Page 13: I understand that it might cause problems if pollution reduction goals are mentioned 

since this introduces altruism towards those who bear these costs, but it seems overly vague to 

mention cost increases without describing the “changes required under pollution reduction 

programs.” I wonder if the industries that would be affected could be mentioned?

Page 13: “inside and outside” seems to flow better than “outside and inside.”

Page 13: A question is needed to allow respondents to gather their thoughts. For example, how 

likely do you think it is that pollution reduction programs would result in higher costs for your 

household? ... very likely, somewhat likely, somewhat not likely, not likely at all.

Page 14: Respondents should either be “choosing” a program or “voting for” it. I think voting

language would be best -- at the bottom of page 15 respondents are asked for their vote and page

16 asks for a vote. If this is preferred then the instructions at the bottom should say “If you want

to vote for Program A (B), mark this box.

Page 15: DO needs arrows (and % decrease) as suggested above.
Page 15: 40% of the bay areas will meet DO goals (strike “will” if this is the current condition).

Page 16: Why shouldn’t respondents think about other households’ welfare? Real life votes

allow consideration of altruism.

Page 21: the 1-7 scale needs to be formatted so it is clear that the numbers match the text (i.e., I

think the numbers need to be centered in the table so that they are directly underneath the text).

Page 22: Future generations are included, so “preserving the environment for current 

generations” should also be included.

Page 23: Why is not italicized? Is this necessary?

Page 23-25: Why is the SD to SA on a 7 point scale here where it is on a 5 point scale earlier

(page 7)?

Page 26: Why does it matter if the partner is “long-term”? Many marriages aren’t long term. 

Page 27: The education categories should be separated .... “some college credit” and “college

degree” are very different levels and should be different answers, same for “some grad school”

or “graduate degree.”

Version A:

Page 9: DO needs a “no change” similarly to water clarity and aquatic grasses.

Version B:

Page 8: “environmental goals for improving the quality of the Chesapeake Bay” is slightly 

different from “environmental goals for water quality in the Chesapeake Bay.” While water 

clarity is used in both the inputs and endpoints versions, the bullet points are different. They 

should be identical.

Page 8: Question #7 raises questions about the species affected with the last phrase “for any ...

species?”

Page 9-11: It would be helpful to explicitly mention that conditions will remain constant with 

current programs.

It seems that the endpoint versions should include striped bass and recreational fish and oysters

and crabs could be bundled as a “shellfish” endpoint.

3. Adequacy of background information to enable the respondents to answer the choice 

questions. Is the information necessary, sufficient, and accurate?
In general the background information covers the topic well. The only glaring omission is full 

coverage of the goals of the policy.

4. Does the survey provide appropriate and sufficient introductory and debriefing questions to, 

for example, identify protest responses and symbolic responses, test retention and understanding 

of information, elicit non-use values, etc.? Are there any additional introductory or debriefing 

questions you feel may be useful for the analysis?

See suggestions above under #2.

I don’t consider identification of protests an important concern. Real-life protest votes can’t be

identified and discarded.

5. Does the survey place sufficient and appropriate emphasis on the budget constraint facing 

respondents, and on potential substitute uses of their budget?

Yes, the survey includes standard language about budget constraints and substitutes.

6. Considering all four versions of the survey, are the program options under consideration 

appropriately and sufficiently characterized? For example, given the focus group summary 

provided, are the appropriate attributes included in the choice experiment?

The problem with the endpoint version is that a large number of attributes should be included. 

That these are not included so as to keep the number of attributes equal to the input version is not 

a sufficient reason. This is an artificial constraint imposed by experimental design. I have serious 

doubts about the validity of the endpoint version.

7. Regarding the “Input” version of the survey, are the links between attributes in the program 

options (such as Dissolved Oxygen) and aspects of the Bay respondents may directly care about 

appropriately described? Are there ways to improve the description of those links?

The descriptions are adequate.

8. Does the survey incorporate appropriate language to ameliorate symbolic responses and 'yea 

saying'?

Symbolic responses are a problem when the good or service is not adequately described. That is 

not a problem with this survey. Yea saying is typically a problem with iterative bidding 

contingent valuation. I would not be especially concerned about it with this survey.

9. Does the survey take appropriate steps to minimize hypothetical and strategic bias, including 

sufficient steps to emphasize consequentiality?

The survey needs to ask a consequentiality question such as “how likely do you think these 

results will be considered by policy makers?” very likely, etc.
Also, sensitivity analysis could be conducted over hypothetical bias by asking respondents how 

certain they are that they would vote for the policy relative to the status quo. For example, for 

those who voted for Program A or B at least once, “When you were voting for Program A or B, 

were you very sure, somewhat sure, or not sure at all that you would actually vote for the 

program over the status quo?” Those who are not very sure could have votes recoded to the 

status quo and the model re-estimated to produce lower values (and less likely to have 

hypothetical bias).

10. Is the format of the choice questions appropriate according to current standards in the stated 

preference literature (e.g., allowing for a status quo option, encouraging neoclassical tradeoffs, 

order and presentation of attributes, etc.)

Yes.

11. Is there evidence of fundamental or critical flaws in the survey? Such as bias in the survey or 

presented information?

There is no evidence of flaws or bias in the survey.

12. Are the proposed attribute levels and use of a constant versus declining baseline appropriate 

for estimating the benefits of the TMDL?

Yes, the constant versus declining baseline will provide a scope test and allow flexibility for 

understanding the actual impacts under existing programs. The models should be estimated with 

constant and baseline versions separately.

13. Is the sampling plan and proposed geographic stratification appropriate for estimating the 

total benefits of the TMDL, including nonuse values?

The sampling plan makes good sense although I have questions about survey mode. It seems like 

the questionnaire is designed for a mail survey. This seems odd given the vast improvements in 

internet survey technology. I would be very concerned about survey response rate. Are monetary 

incentives being given to respondents?
IV. Response from Dr. Robert J. Johnston Director, George Perkins Marsh 

Institute and Professor, Department of Economics, Clark University

Peer Review – Stated Preference Estimation of Benefits from Chesapeake Bay TMDLs

Benefit estimation for the Chesapeake Bay TMDL is challenging. NCEE has made good 

progress in a on a stated preference survey to estimate at least a portion of these benefits. 

Nonetheless, some improvements are required if this survey is to provide valid and defensible 

welfare estimates. Recognizing the controversy surrounding the use of stated preference 

methods for benefits estimation and the high profile of the proposed Chesapeake Bay TMDL, it

is likely that this survey and its results will undergo very careful scrutiny. The goal of this peer 

review is to help ensure that the survey and implementation methods are of sufficient validity, 

quality and defensibility to withstand this scrutiny.

Stated preference design requires a combination of creativity and science, and even experienced 

practitioners sometimes disagree over the most effective design elements. Hence, NCEE should 

consider these comments within the context of other input and review received on the survey. 

The goal of these comments is to be helpful rather than critical, and I hope that NCEE is able to 

use this peer review in its attempts to develop the best stated preference instrument possible.

General Comments

Review of the stated preference surveys, supporting documentation and focus group / cognitive 

interview results in the peer review package suggests that progress has been made towards a 

workable stated preference instrument. Good efforts have been made to link the stated 

preference scenarios to measurable outcomes, and the surveys have been subject to focus group 

testing. Many pages of the survey borrow design elements from our prior choice experiment 

questionnaire (Rhode Island Rivers: Migratory Fishes and Dams) that estimated willingness to 

pay of Rhode Island residents for options to restore fish passage (Johnston et al. 2011a,b; 2012a; 

Schultz et al. 2012).

The proposed survey instrument and methods also face a number of challenges. The policy 

change whose benefits are to be estimated, the Chesapeake Bay TMDL, presents a challenging 

context for welfare estimation. Among the aspects which complicate welfare estimation are (1) a 

very large affected area, (2) myriad complex biophysical effects potentially linked to utility and 

willingness to pay (WTP) in different ways, (3) heterogeneous ecological effects across tidal 

versus fresh waters, and (4) difficulty modeling or predicting the ecological outcomes which 

influence utility.

Within this challenging context, the reviewed surveys require some improvements to ensure 

valid welfare measurement. Most importantly, both economic theory and empirical evidence 

from focus groups reported in the peer review documents indicate that the “Inputs and Dual” 

version of the survey will not generate valid welfare measures. If NCEE insists on using such an 

approach (see detailed discussion below), it is incumbent upon the Agency to provide a clear 

structural utility framework that demonstrates how it is possible for this approach to generate 

valid welfare estimates in the absence of detailed and quantitative information on biophysical 

production functions. In addition to this fundamental issue, there are other issues that should be 
addressed before survey implementation. For example, many key policy attributes remain 

ambiguous, as suggested both by an objective review of the survey instrument and by comments
of focus group and cognitive interview respondents. Moreover, decisions regarding which 

attributes to include and exclude from the survey do not appear to be grounded in an appropriate 

structural model that clarifies linkages between ecological outcomes and utility.

Considering these and other elements described below, I believe that the current endpoint 

versions of the survey could be improved so as to allow valid welfare estimation for many of the 

primary TMDL impacts. The surveys in the peer review package are good start. However, the 

required changes to these surveys are not trivial. Greater specificity should also be given to the 

theoretical and conceptual models underlying the various survey versions.

Additional details, along with specific responses to the peer review questions, are provided 

below.

Peer Review Responses

1. Extent to which the survey results are likely to provide welfare estimates of improving water 

quality in the Chesapeake Bay. Specifically,

1a. the ability of the survey to provide total values

As noted above, there are multiple ways that the Chesapeake TMDL might influence benefits 

realized by different user and nonuser groups throughout affected watersheds and beyond. Some 

of these may be realized through observable changes in consumptive uses (e.g., fishing, 

crabbing), others may be realized though changes in recreation or other behaviors, and still 

others may be related to nonuse or other motivations. Many, but not all of these will be captured 

by the stated preference survey and sampling frames as currently structured.

The current survey instruments seek to estimates the WTP of households for a small set of 

ecological outcomes. Many likely welfare-relevant outcomes such as effects on striped bass and 

recreational fish were “dropped to keep the number of attributes on both versions the same and to 

reduce cognitive burden.” Effects on bird populations were dropped because on an inability to 

model changes, even though “focus groups indicated that bird populations were important.” In 

some cases, the stated rationale for dropping variables has unclear justification. Variables seem 

to have been dropped more for convenience than due to any formal scientific rationale. For 

example, NCEE provides no stated theoretical or empirical reason why it is necessary to “keep 

the number of attributes on both versions the same,” although one might speculate that it is 

related to a desire to minimize the effort of experimental design. There are numerous empirical 

analyses and comparisons of stated preference results within the published literature in which the 

number of attributes differs across contrasted survey versions.

Regardless of whether these design choices are justified, the result is that welfare gains 

associated with such changes cannot be estimated by the survey. Past choice models 

demonstrate that changes to fish and birds in such situations are relevant to individuals’ use and 

nonuse WTP, in addition to welfare effects associated with other biophysical changes to 

measures such as water quality (e.g., Bateman et al. 2006; Hanley et al. 2006a,b; Johnston et al.

2002, 2005; Johnston et al. 2011a,b; 2012a,b; Milon and Scrogin 2006; Pattinson et al. 2011;
among many others). If changes to these species are non-zero, any resulting welfare gain will be 

overlooked by the survey as currently designed. A choice modeling survey cannot provide valid 

WTP estimates for attributes that are not included in survey scenarios. By dropping potentially 

important attributes such as those quantifying effects on bird species—a likely source of benefits 

from the TMDL—the analysis will likely overlook a portion of benefits.

In addition, as currently structured it is unclear how and whether the survey could provide valid 

benefit estimates associated with changes in commercial harvests of crabs, oysters, etc. A 

separate model using commercial fisheries data would be required to estimate these benefits. 

Finally, due to the geographic sampling scale, any benefits realized by Chesapeake Bay visitors 

or those holding nonuse values for TMDL outcomes, yet residing outside of east coast states, 

will not be estimated by the survey. See additional comments on sampling below.

For these and other reasons detailed below, it is unlikely that the reviewed stated preference 

surveys will provide comprehensive estimates of total values from the TMDL. It is possible that 

the endpoint versions of the surveys, if revised appropriately, might provide valid estimates of 

total (average) household WTP for the particular set of ecological attributes included in the 

survey, and the particular set of households sampled. Moreover, some welfare effects that would 

currently be overlooked by the survey could be incorporated through a somewhat more 

comprehensive set of choice scenarios, including some key attributes that are currently omitted.

1b. appropriateness of the proposed choice experiment survey for the estimation of marginal 

willingness to pay for individual attributes

The choice experiment surveys and proposed approaches included in the peer review package are 

a good foundation, but require revisions in order to ensure valid marginal WTP estimates for 

individual attributes. As these methods are currently presented, the resulting marginal WTP 

estimates are unlikely to be valid. This is particularly true for the “Inputs and Dual” versions of 

the survey. Three concerns are paramount. First, there is a potential concern with benchmarking 

all attributes with regard to policy goals of unknown origin and reliability. Second, a number of 

the survey attributes appear ambiguous, both objectively and from the perspective of pretest 

respondents. Responses of focus groups and cognitive interviews—even those at the end of the 

design process—suggest continued difficulties with attribute interpretations, and confusion 

among a non-trivial proportion of respondents. Third and most importantly, it is unclear how 

respondents can provide welfare-revealing responses to the “Inputs and Dual” versions of the 

survey without unsupported speculations regarding final outcomes (or endpoints) that influence 

utility. Each of these concerns is detailed below.

Using Policy Goals as a Reference Condition: The survey attempts to benchmark attribute 

changes to a measurable reference condition, thereby giving respondents additional capacity to 

comprehend scope; this is a positive aspect of the design (Johnston et al. 2012a). However, the 

use of rather ambiguously characterized “policy goals” as an objective benchmark for attributes 

appears problematic. Within such a survey, one could possibly alter estimated WTP solely 

simply by changing the stated goal of restoration. Given that policy goals are endogenous to the 

policy process, heterogeneous across policy makers, subject to political pressure, etc., it is 

unclear how they represent a clear and unambiguous benchmark for scope in a stated preference
survey. Although focus group participants often “like” being told what scientists recommend 

(and there is nothing wrong with doing so in general), this does not imply that doing so alone 

provides a sufficient objective basis for understanding policy scope. How does the 

recommendation of 185,000 acres of aquatic grasses compare to historical levels? Is this a lot or 

a little? Moreover, unless there is universal scientific consensus on these goals, NCEE risks 

having ultimate welfare estimates challenged as a mere artifact of the particular set of goals 

stated in the survey. Whose policy goals are these? Much preferable would be a benchmark 

based on something more objective and grounded; for example historical levels of an attribute or 

ecological reference conditions as discussed in Johnston et al. (2012a) and Schultz et al. (2012). 

Although even ecological reference conditions can be subject to some disagreement, they are 

much more objective and stable than policy goals, which can often change dramatically due to 

factors that have nothing to do with objective ecology.

More fundamentally, how does benchmarking an ecological improvement to someone’s policy 

goal help respondents predict impacts on their future welfare? A clearly stated ecological 

reference condition provides respondents with a means to compare policy changes to a “pristine” 

or “best possible” ecological system, as measured by the condition of comparable nearby 

systems with less or no human influence. Often, respondents will have directly experienced such 

reference systems, providing them with a better capacity to understand the scope of ecological 

changes described in the survey. This is the case in Johnston et al. (2011a,b; 2012a). These 

advantages do not apply to a policy goal. It is unclear how a policy goal provides respondents 

with any additional capacity to predict policy effects on their own welfare. If anything, 

comparing effects to a policy goal might encourage symbolic responses, in which the “policy 

goal” is used by respondents to determine whether we are “doing enough to protect the 

environment,” regardless of concrete effects on their individual welfare.

Potential Attribute Ambiguity: A number of the attributes in the presented surveys come across 

as ambiguous. For example, what is a “DO Goal” and what does it mean? This relates both to 

the meaning of the attribute and to the input/endpoint problem detailed below. How do “DO 

Goals” relate to actual or historical ecological condition? Even as a relatively informed 

researcher, I am not sure how an attribute such as % of the Bay meeting DO goals would affect 

my personal welfare. I wonder how a layperson can make this assessment.

Moreover and more concretely, when describing effects on Blue Crabs and Oysters, the survey 

never defines the numbers used; we are not told whether the numbers presented in survey 

scenarios are average populations, annual harvests or something else. For example, we are told 

that there is a “goal” of 415 million crabs and 35,000 tons of oysters. Nowhere is it stated 

whether this is a harvest goal, a population goal, or perhaps something else. Is 415 million crabs 

the goal for annual crab harvest, or for a steady state crab population? The same ambiguity 

applies to all related changes in choice scenarios – are these changes in populations or harvests? 

Numbers such as these are most often used to denote harvest (e.g., 35,000 tons), but discussion 

early in the peer review documentation suggests that they may be population forecasts instead. 

Either way, the meaning of this attribute depends on such information, which is not provided in 

the survey. These details require clarification; as currently designed the definition of these 

attributes is ambiguous. The fact that respondents appeared to have overlooked this ambiguity 

during focus groups and interviews makes we wonder whether the survey is being read closely
during pretests, and whether respondents are seriously considering the interpretation of choice 

attributes.

For water clarity, NCEE might consider a simple illustration to characterize what is meant by a 

certain number of feet of average visibility. There are a variety of ways to accomplish this, 

ranging from a simple illustration of a secchi disk at different depths to a more complex diagram 

such as that in Bateman et al. (2011). Pretests we conducted for other surveys (Johnston et al.

2012b) suggested that visual cues were often necessary for individuals to truly comprehend 

clarity measures and avoid symbolic responses.

Problems with Ecological Inputs as Attributes in a Choice Experiment: The problems with the 

use of intermediate inputs as attributes in stated preference surveys used to value ecosystem 

outcomes, in the absence of final outcomes, are well-established (e.g., Boyd and Krupnick 2009; 

Johnston et al. 2011a,b; 2012a,b,c; Johnston and Russell 2011; Kontogianni et al. 2010; Limburg 

and Folke 1999; US EPA 2009). A primary concern is that lack of understanding of the role of 

intermediate services will lead individuals to state a WTP for these services that does not 

appropriately reflect the role these services play in providing final ecosystem services.

The exclusion of final services (or endpoints) and inclusion of intermediate processes (or inputs) 

within stated preference survey scenarios may bias welfare estimates for at least two reasons 

(Johnston et al. 2011a,b; 2012a,b,c). First, respondents asked to value changes in inputs may not 

be aware of resulting impacts on welfare-relevant endpoints, leading to WTP estimates that do 

not fully reflect welfare contributions of the intermediate processes. Second, even if respondents 

recognize a relationship between the input and endpoint, they may have an incorrect 

understanding of the biophysical relationships that link the two. Survey responses and resulting 

welfare estimates will be based at least in part on these incorrect understandings. As noted by 

Carson (1998, p. 23), stated preference “[r]espondents will tend to fill in whatever details are 

missing in the ... survey with default assumptions. These may differ considerably from what the 

researcher perceives.” This is particularly true for ecological resources and processes for which 

respondents have little baseline information and for which the problem of defining endpoints is 

critical (Bateman et al. 2011; Kontogianni et al. 2010).

Simply put, to answer stated preference survey questions when information on a final service is 

omitted, respondents must speculate about the effects of related intermediate outcomes on their 

welfare. Without specific and quantitative information on biophysical production functions, 

respondents will be unable to correctly predict effects on welfare-relevant final services (Boyd 

and Krupnick 2009; Kontogianni et al. 2010). Awareness of this problem is not limited to 

economists; ecologists and others have also explicitly noted the lack of validity of stated 

preference welfare estimates associated solely with intermediate inputs, when respondents lack 

the capacity to link these to final welfare-relevant endpoints (e.g., Limburg and Folke 1999).

The following simple theoretical illustration, quoted from Johnston et al. (2012c), illustrates the 

problem:

“Suppose household has a utility function of the form, , where

and , and and are
measurable ecological processes or conditions. Both and are direct 

arguments in Uh(·); they do not require further ecological production or 

transformation in order to influence human benefits, and are hence classified as 

final ecosystem services. In contrast, Z is an intermediate service. It requires a 

further influence on Y through ecological production function Y(·) to influence 

Uh(·). This simple framework also illustrates that an ecological outcome X can in 

principle affect utility directly (and hence be a final service) as well as indirectly 

through its contribution to the production of another final service Y.

The marginal utility of a change in X is calculated as

reflecting both the direct and indirect effect of the 

change. From a theoretical perspective, WTP estimated using stated preference 

methods provides a money metric of this change. However, a survey respondent 

without ecological expertise might not be aware of the true relationship Y(X, Z) 

through which changes in X influence Y. If a stated preference scenario were to 

omit information on Y (a final ecosystem service) yet include information on X 

(the related intermediate service), respondents might condition responses on 

erroneous speculations regarding the true ecological production function Y(X, Z). 

That is, they might speculate an incorrect relationship between X and Y. Survey 

responses and resulting welfare estimates would reflect this incorrect 

assumption.”

This seems to be exactly what is done in the “Inputs and Dual” versions of the survey that 

describe policy outcomes in terms of Dissolved Oxygen goals. It seems quite unlikely that 

dissolved oxygen itself is a direct argument in utility for most respondents – it is primarily an 

input or intermediate outcome contributing to other valued final outcomes. This notion is 

expressed clearly by focus group and interview respondents. Yet the survey provides no 

quantitative biophysical information that would enable respondents to translate dissolved oxygen 

into utility-relevant endpoints (things they care about).

Confusion with the input version of the survey is evident in focus group and cognitive interview 

results. Even as late as the final cognitive interview sessions, multiple subjects expressed 

difficulties with the dissolved oxygen attribute, in addition to one subject that “found the choice 

questions difficult and confusing and ultimately did not make decisions based on attribute 

levels.” The final focus group session similarly indicated that “it was not clear to them how 

inputs related to potential endpoints they care about.” This is the type of reaction predicted by 

economic theory summarized above, and makes it unlikely that the “Inputs and Dual” versions of 

the survey will provide valid estimates of WTP.

Given these fundamental theoretical and empirical problems, NCEE’s plans to “rely primarily on 

the input version for the benefit analysis” (p. 1-6) appear fundamentally flawed. This will lead to 

welfare estimates with unknown and potentially severe biases. The primary motivation for this 

choice appears to be the difficulty generating forecasts of blue crab and oyster populations. This 

difficulty is insufficient justification for a flawed choice modeling approach that uses inputs 

within a stated preference scenario, with an almost total lack of biophysical information from 

which respondents might predict welfare-relevant outputs.
There are ways around this problem using a more appropriate endpoint version of the survey. 

Choice modeling enables NCEE to specify a range of reasonable outcomes for relevant 

endpoints, and calculate benefits contingent upon different possible outcomes. If ecological 

forecasts of sufficient certainty are not available for these endpoints, then a benefit sensitivity 

analysis can be conducted, much as NCEE has planned for the constant vs. declining baseline 

versions of the survey. This would be much preferable to the use of the “Input” version of the 

survey which has been proposed, which will lead to welfare estimates of unclear validity and 

defensibility.

1c. appropriateness of a choice experiment survey format for the elicitation of preferences in 

this case (i.e., compared to alternatives such as contingent valuation).

The choice experiment format is generally appropriate for elicitation of preferences in this case, 

and is particularly appropriate here given uncertainty regarding the final ecological outcomes of 

the TMDL. The choice experiment format is explicitly designed such that the model can 

incorporate a wide-range of potential outcomes, enabling WTP estimates for a wide range of 

possible outcomes of the TMDL. Unlike older forms of stated preference analysis in which 

WTP is contingent upon a single specification of ecological effects (e.g., contingent valuation), 

choice experiments solicit responses under a large number of possible attribute configurations. 

This is well suited to a case such as this one, in which the final ecological outcomes may be 

difficult to predict ex ante.

As an aside, one possible advantage of contingent valuation in this context is that it would avoid 

the need to disaggregate policy outcomes into a small set of individual attributes. From the peer 

review documents, it appears that this has been a challenge. Contingent valuation, while lacking 

the flexibility of contingent valuation, would help mitigate this problem. Nonetheless, even 

considering this, I believe that the advantages of choice experiments outweigh the disadvantages 

in this case.

2. Overall quality of the survey.

As implied by the comments above, the currently presented draft surveys provide a solid 

foundation upon which to build. There has clearly been a lot of work put into the design of these 

surveys thus far. However, based on a review of the draft surveys and the most recent rounds of 

focus groups and interviews, it does not appear that all design shortcomings have been 

addressed. When addressing overall survey quality, the following section does not repeat the 

issues raised in sections above. Rather, it focuses on concerns that have not yet been raised. 

These issues are numbered below.

1. Within the final set of cognitive interviews, one out of four respondents (25% of tested 

subjects) was unable to make choices based on presented attribute levels. An additional 

two subjects (50% of the subjects) did not understand at least one of the choice attributes. 

All subjects appeared to suggest substantive changes to the survey. Within the final 

focus group, “respondents found water quality scores difficult to comprehend and 

evaluate,” and similarly suggested a number of changes to the survey. All of the later 

groups and interviews seem to have requested additional information on the policies and
outcomes to be evaluated. These results suggest that the surveys, while displaying many 

positive aspects, have not yet reached the point where they are ready to be implemented.

2. Other potential concerns are revealed by a review of the peer review materials. In 

general, the survey appears to be quite long at approximately 28 pages (including the 

cover). Many focus group and interview respondents comment on the excessive length of 

the survey. In contrast, the choice experiment survey used by EPA/OW to estimate WTP 

for 316(b) policy impacts was approximately 20 pages in length (again including the 

cover). The surveys reported in (Johnston et al. 2011a,b; 2012a,b) are similarly no longer 

than 20 pages. A survey developed by Paul Jakus to evaluate policies that would 

similarly reduce nutrient loading in Utah’s lakes and rivers is only 8 pages. This reflects 

survey lengths common in choice experiments; lengths between 12 and 20 pages are 

common. I worry about response rates and respondent attention to such a long survey. 

What is the length of time required to complete this instrument (presumably this was 

observed during focus groups)? Is there evidence that respondents outside of focus 

groups are willing to complete a 28 page survey? A small pilot survey might be justified 

to address such questions before fielding a large scale final survey.

3. The survey also includes long passages of text, often uninterrupted by graphics or other 

more visually appealing elements. Is there evidence that respondents are reading and 

fully comprehending this text? For example, the fact that respondents seemingly missed 

the ambiguity regarding the definition of the blue crabs and oysters attribute (i.e., are they 

harvest or population measures?) makes we wonder the extent to which they are carefully 

considering all survey questions and understanding the definitions of all attributes. There 

are also errors in choice scenarios that were not noticed by respondents (see discussion in 

subsection 6 below), that further suggest a lack of attention and comprehension.

4. Although I assume that this will be addressed before fielding the survey, the survey 

graphics and design currently come across as bit unpolished. Graphical problems include 

inconsistent spacing between Likert Scale categories (numbers) in many questions that 

becomes evident when surveys are printed. There are also graphical and textual 

inconsistencies across questions. For example, the formatting of question 16 (page 23 in 

most surveys) includes a table of all one color, with text center justified in the first 

column. Questions 17 and 18 on the following pages are two different colors, with the 

text left justified in the first column. While each of these issues alone is minor, together 

they influence the overall polish of the survey.

The graphical choices of the survey are also seemingly inconsistent with the survey 

content and could lead to speculation and confusion. For example, the cover photo shows 

an osprey nest. Yet as noted above the survey does not discuss or seek to value changes 

to bird populations that would occur as a result of the TMDL. Why does the survey 

cover illustrate a large photo of a resource that is not addressed by the survey (effects on 

birds have been omitted as described above)? This might invite symbolic responses.

5. The experimental design described on page 1-6, while acceptable, might not be the most 

efficient for the intended application. While not clearly specified in the peer review
document, the design appears to be an orthogonal, fractional factorial main effects plan 

designed for a linear model covariance matrix. If this is an accurate interpretation, such a 

design does not take advantage of recent methods that allow the creation of designs that 

are tailored to the covariance matrix of a choice model, e.g., through a minimization of 

D-error (e.g., Kuhfeld 2010; Rose and Bliemer 2009, 2012), and are therefore more 

efficient for choice experiments. These designs show that greater efficiency can be 

obtained in choice model designs through the sacrifice of strict orthogonality. Also, for 

relatively small designs such as this one, orthogonal blocking (of questions among survey 

versions) can often provide superior results to a simple “random” blocking, which does 

not necessarily maximize orthogonality between attribute levels and blocking. The 

experimental design is fine, but could be improved.

A similar issue was confronted in EPA/OW’s 316(b) stated preference survey. A parallel 

main effects design was applied for a variety of practical reasons (e.g., the time allowed 

to finalize the design, desire for a compact design, etc.). This allowed for relatively 

efficient and unbiased estimation of main effects. Yet in retrospect it might have been 

helpful to have an experimental design that allowed for the estimation of interactions—

particularly given that NCEE has requested various tests that can only be implemented 

through the use of experimental designs that allow for the estimation of interactions (see 

related discussion in sub-point 7 below). NCEE could avoid such problems by 

implementing somewhat more complex designs from the start.

6. There appear to be errors in the choice questions. For example, on page 19 in the “Inputs 

and Dual,” Constant Baseline version, Program B states that 80,000 acres of Aquatic 

Grasses would result, and describes this as a 50% increase. Yet 80,000 acres is also the 

status quo. So there is no increase. That is, if 80,000 acres is the status quo, then 80,000 

acres cannot also represent a 50% increase from the status quo. This same error carries 

through to all of the “Inputs and Dual” versions of the survey. A similar error appears in 

the water clarity attribute: 5 feet of visibility in Program B is described as “No Change” 

from the status quo of 3 feet of visibility. Although these errors can be easily corrected, 

the fact that focus group and interview participants did not notice them suggests that 

respondents may not be carefully considering attribute levels in choice scenarios. There 

appears to be no evidence that focus group or interview respondents noticed these 

seemingly important errors. What does this say about respondents’ comprehension of 

attribute levels in the choice questions?

7. Two recent stated preference surveys have been developed by US EPA for analysis of 

benefits related to policy changes affecting aquatic ecosystems: (1) the survey developed 

by EPA/OW for 316(b) regulatory analysis, and (2) the present survey developed by 

NCEE for the Chesapeake Bay TMDL. Over the past year NCEE has requested a large 

number of validity tests and design procedures for the EPA/OW 316(b) survey. Although 

I assume that NCEE plans to accommodate similar procedures for the present 

Chesapeake Bay study, few of the required design elements appear to have been 

incorporated thus far in the Chesapeake Bay study design. To be consistent, the present 

study should accommodate the same specifications, procedures and validity tests that 

have been suggested for the EPA/OW 316(b) analysis. These includes accommodation
for a variety of tests of external scope, question ordering, adding-up, and response 

heuristics (for the most recent example, see the memorandum “2012 06 28 Game plan for 

validity tests” sent to EPA/OW by NCEE). To be clear, I am not suggesting that all of 

these tests are necessary or even appropriate, but only that NCEE should apply the same 

tests and standards to both survey efforts.

3. Adequacy of background information to enable the respondents to answer the choice 

questions. Is the information necessary, sufficient, and accurate?

The background information provided by the survey provides a good perspective on TMDL 

effects. I am concerned, however, by a seeming lack of consistency between the background 

information provided by the survey and the later valuation tasks (choice questions) presented to 

respondents.

For example, much of the introductory text in the survey emphasizes effects on “wildlife 

habitats” and “recreation opportunities” (for example, see question 3 and the box above it; also 

see the top of page 6). This seems to imply that the valuation questions will address effects on 

wildlife and recreation. Yet, the input version of the survey includes no attributes that convey 

effects on wildlife or recreation. The endpoint choice questions, while communicating effects on 

two aquatic wildlife species (oysters and blue crabs), include no mention of recreation. There 

seems to be a bit of a disconnect between language earlier in the survey and the composition of 

choice questions later on. Why emphasize something such as recreation, when it is not 

addressed in the valuation scenarios (except perhaps indirectly through water clarity impacts).

Such inconsistencies are common throughout the survey. For example, page 6 characterizes a 

large number of relatively imprecise effects such as “more enjoyable boating and swimming”, a 

“cleaner and healthier aquatic environment”, etc. These impacts are not addressed within choice 

questions. When answering choice questions, does NCEE expect respondents to consider effects 

such as “more enjoyable boating and swimming” and a “cleaner and healthier aquatic 

environment”? If no, why are they emphasized in the introductory text? If yes, why are they not 

addressed in the choice questions? The presence of these descriptions on page 6 implies that 

these effects are important to some people (i.e., are welfare-relevant); this belies their absence in 

choice questions. Should respondents consider these effects when answering choice questions? 

If so, what outcomes should they assume? On what should they base these assumptions? 

Currently, the survey provides no quantitative biophysical information that would enable 

respondents to quantify effects such as these.

This inconsistency between background information presented early in the survey and questions 

asked later on is reflected directly in the responses of focus group and interview respondents. 

For example, within the final cognitive interview set, “most respondents suggested it would be 

useful to have information early in the instrument to describe what was going to be asked later in 

the survey.” This suggests that more work is needed to ensure that the introductory and 

background information provides sufficient preparation for respondents to answer choice 

questions.

As a final note, all of the introductory and background material focuses respondents on the 

importance of the Chesapeake Bay. There is no discussion of other non-environmental policy
priorities that might be considered substitute uses of a respondents budget. Hence, skeptics 

might challenge the survey on the grounds that it fails to mention substitute commodities or 

programs.

4. Does the survey provide appropriate and sufficient introductory and debriefing questions 

to, for example, identify protest responses and symbolic responses, test retention and 

understanding of information, elicit non-use values, etc.? Are there any additional 

introductory or debriefing questions you feel may be useful for the analysis?

The survey includes a reasonable set of questions to identify protest and symbolic responses. 

There are a few questions which might be of limited usefulness, depending on the models which 

are being considered. For example, it is unclear how NCEE plans to use responses to question 

two, which relates to the frequency of visiting lakes, rivers or tidal bays to engage in recreational 

activities.

There are other question types that are currently missing from the survey that would provide 

useful information if included. For example, it would be useful to include a certainty question of 

the type discussed by Ready et al. (2010). In addition, certain types of “warm-up” questions that 

are typically included in surveys of this type are missing from the current survey. These include 

questions addressing the importance placed on Chesapeake Bay condition relative to other 

important policy priorities to which the government might allocate resources (see discussion of 

introductory materials above).

There are few questions that truly and objectively “test retention and understanding of 

information.” If this is desired, additional questions will be required.

5. Does the survey place sufficient and appropriate emphasis on the budget constraint facing 

respondents, and on potential substitute uses of their budget?

Emphasis on the budget constraint and substitute commodities, while generally satisfactory,

could be improved. As noted above, there is no discussion of other non-environmental policy 

priorities that might be considered substitute uses of a respondents budget. The only clear 

statement of these tradeoffs is found on survey page 16: “Paying the costs means you would 

have less money to spend on other things such as food, clothes, going on trips, and even towards 

resolving other environmental problems you care about.” While this is an appropriate statement, 

it does not reflect potential tradeoffs involving non-environmental government programs for 

things such as education.

Also, the consequentiality of survey choices could be better highlighted, particularly with regard 

to effects on households’ budgets. For example, the statement found on page 16, “We urge you 

to respond as though costs for your household really would go up if the program were 

implemented,” is weak and unconvincing. Why not say something stronger and more direct such 

as “Costs for your household will really increase if programs such as these are implemented, and 

your answers to this survey will help determine whether this happens.” Given that this survey is 

being implemented by US EPA to inform regulatory policy, the survey could incorporate much 

stronger language to highlight consequentiality.
6. Considering all four versions of the survey, are the program options under consideration 

appropriately and sufficiently characterized? For example, given the focus group summary 

provided, are the appropriate attributes included in the choice experiment?

No, the program options under consideration are not appropriately and sufficiently characterized 

in all four survey versions. Comments relative this question are provided in sections above. 

Problems include fundamental flaws with the “Inputs and Dual” versions of the survey; these 

flaws are highlighted by focus group and interview responses as identified above. In addition, 

many choice attributes remain ambiguous, both from an objective biophysical standpoint (e.g., 

do crab and oyster numbers reflect harvests or populations?), and based on the comments of 

focus groups and cognitive interviews. Even as of the final focus groups and interviews 

respondents were expressing confusion with attributes such as dissolved oxygen, with one out of 

four respondents in the final cognitive interview unable to answer choice questions based on the 

attribute levels provided, and an additional two out of four unable to understand the dissolved 

oxygen attribute. Respondents throughout many focus groups showed evidence of the 

speculation discussed by Johnston et al. (2011b, 2012a,b,c) when faced with inputs rather than 

endpoints. For example, in cognitive interview set 1, “One respondent had difficulty and wanted 

to have information on endpoints rather than inputs. He answered the questions, but based 

responses largely upon his general knowledge of water quality in the area rather than specific 

input attribute levels.” This is exactly the type of speculation suggested by theory, leading to 

biased welfare estimates. Other potential problems, such as the use of “policy goals” to 

benchmark attribute levels, are discussed above. Finally, the fact that respondents appear to 

have missed multiple errors in choice questions (as described above) further suggests that they 

may not fully understand the choice scenarios.

7. Regarding the “Input” version of the survey, are the links between attributes in the program 

options (such as Dissolved Oxygen) and aspects of the Bay respondents may directly care 

about appropriately described? Are there ways to improve the description of those links?

The ”Input” versions of the survey as currently drafted include no specific, quantitative 

biophysical information that would enable respondents to link attributes such as dissolved 

oxygen to aspects of the Bay that respondents directly care about. Biophysical links are not

adequately described, and certainly not in a sufficient way for respondents to predict effects on 

their welfare. As stated in the peer review documentation (p. 1-6), it is the lack of information 

on these biophysical linkages which caused NCEE to propose the use of the “Inputs and Dual” 

versions of the survey:

“Generating forecasts of blue crab and oyster populations for the endpoint version 

of the survey requires all of the modeling of the input version with an additional

modeling step using CBFEM. That and the fact that CBFEM is based on new

science that contradicts the general result that more nutrients increase biological

productivity in a waterbody, and thus lead to more fish, has led NCEE to rely 

primarily on the input version for the benefit analysis and to administer the

endpoint version mainly for methodological purposes.”

Within the above passage, NCEE admits that it cannot characterize adequately the biophysical 

linkages between “inputs” such as nutrients and dissolved oxygen and the outputs that are 

relevant to households’ utility. Since NCEE itself is unable to quantify these biophysical 

linkages, it is impossible for the survey to adequately characterize them. Simply put, if NCEE is 
unable to forecast utility-relevant endpoints based on the inputs provided in the “Inputs and

Dual” versions of the survey, how can we expect that uninformed respondents will be able to 

forecast these endpoints? If they cannot do so, how can they meaningfully predict their welfare?

As described above, the use of a survey design that quantifies endpoints rather than inputs 

eliminates the need for respondents to speculate regarding biophysical production functions that 

are not presented in the survey. NCEE appears to be using the “Inputs” versions of the survey in 

an ill-advised attempt to eliminate the need to predict the welfare relevant biophysical endpoints 

of the TMDL. While this might obscure the problem, it does not fix it. There is no way for 

respondents to meaningfully express WTP for a policy if they cannot predict how and why that 

policy will influence outcomes that they value.

8. Does the survey incorporate appropriate language to ameliorate symbolic responses and

'yea saying'?

In general, the survey incorporates reasonable language to ameliorate symbolic responses and 

yea-saying. There are some elements of the survey, however, that could also encourage symbolic 

responses. The most direct potential cause of symbolic responses in the present case is the use of 

the “Inputs” versions of the survey; as respondents will be unable to predict welfare-relevant 

endpoints from these versions of the survey, they will be more likely to give symbolic responses. 

Another potential cause of symbolic responses in the present surveys is the broad and generic 

discussion of TMDL benefits earlier in the survey (e.g., “more enjoyable boating and 

swimming”, a “cleaner and healthier aquatic environment”) that are not then reflected in choice 

scenario attributes. This may cause some respondents to answer symbolically, basing responses 

on generic “good things for the environment” that are highlighted earlier in the survey but 

omitted from choice scenarios. Similar problems might be encouraged by such design elements 

as a cover photo showing a charismatic bird species (osprey), when choice scenarios explicitly 

omit effects on bird species. Hence, while the cover implies that effects on birds are present,

respondents are asked to consider policies with no stated effects on birds; the result could be

responses driven either by symbolic motivations or speculations regarding the omitted effects on 

birds.

9. Does the survey take appropriate steps to minimize hypothetical and strategic bias, 

including sufficient steps to emphasize consequentiality?

The survey takes reasonable steps to minimize hypothetical and strategic bias, although language 

concerning the consequentiality of the survey could be strengthened as described above. Greater 

emphasis that the survey is being implemented by the US EPA to help directly inform policy 

could also help; right now the survey presents this information in only rather generic terms. As 

noted above, it would also be useful to include a certainty question of the type discussed by 

Ready et al. (2010), as a means to implement potential post hoc adjustments.

10. Is the format of the choice questions appropriate according to current standards in the 

stated preference literature (e.g., allowing for a status quo option, encouraging neoclassical 

tradeoffs, order and presentation of attributes, etc.).

Aside from the various issues noted above, the general format of the choice questions appears to 

be appropriate. However, as noted above, there are a variety of issues that require attention, 

including errors in the choice questions that appear to have been missed during focus groups (see 

response to peer review question number 2, sub-section 6). Again, this suggests that focus group
and interview respondents were not carefully considering attribute levels, and may not have been 

making appropriate neoclassical tradeoffs.

11. Is there evidence of fundamental or critical flaws in the survey? Such as bias in the survey 

or presented information.

The above sections highlight various flaws that should be addressed. Many of these are 

fundamental to the estimation of valid and defensible welfare estimates. The most fundamental 

and critical of these is the use of the “Inputs” versions of the survey, but there are a variety of 

other improvements that are required.

12. Are the proposed attribute levels and use of a constant versus declining baseline 

appropriate for estimating the benefits of the TMDL?

Notwithstanding the problems with attribute definitions and errors in some choice questions as 

detailed above, the general attribute levels appear acceptable. The constant versus declining 

baseline is also acceptable, although this requires that a full experimental design is conducted 

(and a sufficient number of surveys collected) for each survey version. Also, this will only 

enable benefit estimates to be calculated for a single declining baseline (as presented in the 

survey) – any other baseline will invalidate benefit estimates. A more flexible, though more 

complex experimental design could allow different respondents to view different baselines (with 

the baseline constant within each survey version). This would enable estimation of benefits 

under a range of different baselines. This, however, would require a greater sample size for any 

desired level of precision, and would also require an experimental design that is much more 

complex than the one suggested by NCEE.

13.Is the sampling plan and proposed geographic stratification appropriate for estimating the 

total benefits of the TMDL, including nonuse values?

The proposed sampling and geographic stratification should provide a minimum number of

surveys necessary to estimate choice models (~500 to 600 completed surveys for each of the four 

proposed versions) for the entire sample area, but may not allow models to be reliably estimated 

at smaller geographical scales (e.g., statewide models). The proposed sampling plan will likely 

capture a majority of total benefits, but will not likely allow estimation of total benefits, 

including nonuse values. Given the iconic nature of the Chesapeake Bay, it is very possible that 

some individuals residing outside of east coast states will have nonuse values for TMDL 

outcomes. Also, some individuals who do not reside in east coast states visit the Chesapeake 

Bay area regularly for recreation. Values held by such individuals will not be captured by the 

proposed plan.

References

Bateman, I. J., M. A. Cole, S. Georgiou, and D. J. Hadley. 2006. Comparing contingent valuation 

and contingent ranking: A case study considering the benefits of urban river water quality 

improvements. Journal of Environmental Management 79:221-231.

Bateman, I.J., G.M. Mace, C. Fezzi, G. Atkinson and K. Turner. 2011. Economic Analysis for

Ecosystem Service Assessments. Environmental and Resource Economics 48(2): 177-218.
Boyd, J. and A. Krupnick. 2009. The Definition and Choice of Environmental Commodities for

Nonmarket Valuation. RFF DP 09-35. Washington, DC: Resources for the Future.

Carson, R.T., 1998. Valuation of Tropical Rainforests: Philosophical and Practical Issues in the

Use of Contingent Valuation. Ecological Economics 24(1): 15-29.

Hanley, N., S. Colombo, D. Tinch, A. Black, and A. Aftab. 2006a. Estimating the benefits of 

water quality improvements under the Water Framework Directive: are benefits transferable? 

Eur Rev Agric Econ 33:391-413.

Hanley, N., R. E. Wright, and B. Alvarez-Farizo. 2006b. Estimating the economic value of 

improvements in river ecology using choice experiments: an application to the water framework 

directive. Journal of Environmental Management 78:183-193.

Johnston, R.J., G. Magnusson, M. Mazzotta and J.J. Opaluch. 2002. Combining Economic and 

Ecological Indicators to Prioritize Salt Marsh Restoration Actions. American Journal of 

Agricultural Economics 84(5): 1362-1370.

Johnston, R.J., J.J. Opaluch, M.J. Mazzotta, and G. Magnusson. 2005. Who Are Resource 

Nonusers and What Can They Tell Us About Nonuse Values? Decomposing User and Nonuser 

Willingness to Pay for Coastal Wetland Restoration. Water Resources Research 41(7): W07017.

Johnston, R.J., M. Ramachandran, E.T. Schultz, K. Segerson and E.Y. Besedin. 2012b. 

Disentangling Intermediate and Final Outcomes of Aquatic Ecosystem Restoration: Implications 

for Stated Preference Welfare Estimation. Selected Paper at the Association of Environmental 

and Resource Economists (AERE) Summer Conference, Asheville, NC, June 3-5.

Johnston, R.J. and M. Russell. 2011. An Operational Structure for Clarity in Ecosystem Service

Values. Ecological Economics 70(12): 2243-2249.

Johnston, R.J., E.T. Schultz, K. Segerson, E.Y. Besedin and M. Ramachandran. 2012a. 

Enhancing the Content Validity of Stated Preference Valuation: The Structure and Function of 

Ecological Indicators. Land Economics 88(1): 102-120.

Johnston, R.J., E.T. Schultz, K. Segerson, E.Y. Besedin and M. Ramachandran. 2012c. Stated 

Preferences for Intermediate versus Final Ecosystem Services: Disentangling Willingness to Pay 

for Omitted Outcomes. Selected paper at the Northeastern Agricultural and Resource Economics 

Association (NAREA) Workshop, Economics of Agricultural and Rural Ecosystem Services, 

Lowell, MA, June 12-13.

Johnston, R.J., E.T. Schultz, K. Segerson and E.Y. Besedin. 2011a. Bioindicator-Based Stated 

Preference Valuation for Aquatic Habitat and Ecosystem Service Restoration, in Bennett, J. ed. 

International Handbook on Non-Marketed Environmental Valuation. Cheltenham, UK: Edward 

Elgar, pp. 159-186.
Johnston, R.J., K. Segerson, E.T. Schultz, E.Y. Besedin and M. Ramachandran. 2011b. Indices 

of Biotic Integrity in Stated Preference Valuation of Aquatic Ecosystem Services. Ecological 

Economics 70(11): 1946-1956.

Kontogianni, A., G.W. Luck and M. Skourtos. 2010. Valuing Ecosystem Services on the Basis of 

Service-Providing Units: A Potential Approach to Address the ‘Endpoint Problem’ and Improve 

Stated Preference Methods. Ecological Economics 69(7): 1479–1487.

Kuhfeld, W.F. 2010. Marketing Research Methods in SAS: Experimental Design, Choice, 

Conjoint, and Graphical Techniques. Cary, NC: SAS Institute.

Limburg, K.E and C. Folke. 1999. The Ecology of Ecosystem Services: Introduction to the

Special Issue. Ecological Economics 29(2): 179-182.

Milon, J. W., and D. Scrogin. 2006. Latent Preferences and Valuation of Wetland Ecosystem

Restoration. Ecological Economics 56:162-175.

Pattison, J., P. C. Boxall, and W. L. Adamowicz. 2011. The Economic Benefits of Wetland 

Retention and Restoration in Manitoba. Canadian Journal of Agricultural Economics-Revue 

Canadienne D Agroeconomie 59:223-244.

Ready, R.C., P.A. Champ and J.L. Lawton. 2010. Using Respondent Uncertainty to Mitigate

Hypothetical Bias in a Stated Choice Experiment. Land Economics 86(2): 363-381.

Rose, J.M. and Bliemer, M.C.J. (2009) Constructing Efficient Stated Choice Experimental

Designs, Transport Reviews, 29(5), 587‐617.

Rose, J.M. and Bliemer M.C.J. (2012) Stated preference experimental design strategies, in A. 

Hensher, D.A. (ed.), Transport Economics: Critical Concepts in Economics (Vol. 1), Routledge, 

Oxford, United Kingdom, 304‐32.

Schultz, E.T., R.J. Johnston, K. Segerson and E.Y. Besedin. 2012. Integrating Ecology and 

Economics for Restoration: Using Ecological Indicators in Valuation of Ecosystem Services. 

Restoration Ecology 20(3): 304-310.

U.S. Environmental Protection Agency (EPA). 2009. Valuing the Protection of Ecological

Systems and Services: A Report of the EPA Science Advisory Board. Washington, DC.
Peer Review Package for Stated Preference Estimation of

Benefits from the Chesapeake Bay TMDLs
Peer Review Package for the Stated Preference Estimation of Benefits from the Chesapeake Bay TMDLs

Abt Associates Inc. Contents ▌pg. i

Table of Contents

1. Project Summary ...............................................................................................................1-1

1.1 Background................................................................................................................1-1

1.2 Survey Design............................................................................................................1-1

1.2.1 Attribute Selection........................................................................................1-1

1.2.2 Attribute Metrics and Presentation ................................................................1-3

1.2.3 Information Section ......................................................................................1-4

1.2.4 Payment Vehicle...........................................................................................1-4

1.2.5 Attribute Levels............................................................................................1-4

1.2.6 Question Format ...........................................................................................1-5

1.2.7 Baselines ......................................................................................................1-5

1.2.8 Provision Mechanism....................................................................................1-5

1.3 Experimental Design..................................................................................................1-6

1.3.1 Survey Treatments........................................................................................1-6

1.3.2 Sampling ......................................................................................................1-6

1.4 References .................................................................................................................1-7

1.5 Tables and Figures .....................................................................................................1-8

2. Charge Questions .............................................................................................................2-12

2.1 General Impression of Survey Instrument and Proposed Study .................................2-12

2.2 Survey Instrument....................................................................................................2-12

2.3 Study Design............................................................................................................2-13

List of Figures

Figure 1 Sampling Strata..............................................................................................................1-11

List of Tables

Table 1 Lake Condition Categories.................................................................................................1-8

Table 2 Survey Versions ................................................................................................................1-8
Peer Review Package for the Stated Preference Estimation of Benefits from the Chesapeake Bay TMDLs

Abt Associates Inc. Contents ▌pg. ii

Table 3 Attribute Levels.................................................................................................................1-9

Table 4 Sampling Plan .................................................................................................................1-10

Table 5 Number of Households and Household Sample for Each Study Region............................1-10

Attachments

Attachment A Stated Preference Questionnaire, Attributes are Inputs & Dual, Constant Baseline

Attachment B Stated Preference Questionnaire, Attributes are Endpoints & Dual, Constant Baseline

Attachment C Stated Preference Questionnaire, Attributes are Inputs & Dual, Declining Baseline

Attachment D Stated Preference Questionnaire, Attributes are Endpoints & Dual, Declining Baseline

Attachment E Focus Group and Cognitive Interview Summary
Peer Review Package for the Stated Preference Estimation of Benefits from the Chesapeake Bay TMDLs

Abt Associates Inc. pg. 1-1

1. Project Summary

1.1 Background

On May 12, 2009 the President signed Executive Order 13508 calling for the protection and

restoration of the Chesapeake Bay. In response to the Executive Order and other considerations the

Environmental Protection Agency established Total Maximum Daily Loads (TMDLs) of nitrogen,

phosphorus, and sediment for the Chesapeake Bay. These TMDLs called for reductions of 25, 24,

and 20%, respectively, of these pollutants (EPA 2011).

The Chesapeake Bay watershed encompasses 64,000 square miles in parts of six states and

the District of Columbia. A 2004 report on implementation of the “tributary strategies” proposed

under an earlier plan for Bay restoration estimated their cost at $28 billion in capital costs plus an

additional $2.7 billion dollars per year in perpetuity for operating and maintenance costs (Blue

Ribbon Panel 2004). The watershed states of New York, Pennsylvania, Delaware, West Virginia,

Virginia, and Maryland, as well as the District of Columbia, have developed Watershed

Implementation Plans (WIPs) detailing the steps each will take to meet its obligations under the

TMDL. EPA has begun a new study to estimate costs of compliance with the TMDL. It is important

to put cost estimates in perspective by estimating the corresponding benefits.

EPA’s National Center for Environmental Economics (NCEE) is undertaking a benefits

analysis of improvements in Bay water quality under the TMDL, as well as of ancillary benefits that

might arise from measures taken to meet the required reductions in the Bay. As part of this analysis,

NCEE plans to conduct a broad-based inquiry into benefits using revealed and stated preference

methods. Benefits from meeting the TMDL for the Chesapeake Bay will accrue to those who live

near the Bay or visit for recreation, those who live near lakes and rivers in the watershed or visit them

for recreation, and those who live further away and may never visit the Bay or the surrounding lakes

and rivers but have a general concern for the environment. While benefits from the first two

categories can be captured by the hedonic property value and recreation demand analyses also being

conducted by NCEE, only stated preference methods can capture the nonuse benefits of the latter.

The vast scope of the TMDL impacts, both geographically and ecologically, makes this study

uniquely challenging. Ecological changes in the Bay include aesthetic attributes like water clarity,

color, and odor; increased fish and shellfish populations from higher dissolved oxygen levels; and

general ecological conditions like more aquatic grass habitat, oyster reef habitat, and smaller, less

frequent hypoxic events. The management practices that States will implement to meet the TMDL in

the Chesapeake Bay will also reduce nutrient and sediment loads in freshwater lakes, ponds, rivers

and streams in the watershed. A comprehensive benefits estimate will include the welfare impacts of

these ‘upstream’ changes as well.

1.2 Survey Design

1.2.1 Attribute Selection

Focus groups were conducted to develop the survey instrument. The early sessions focused on

narrowing the list of possible attributes to a set that people relate to and for which changes could be
Peer Review Package for the Stated Preference Estimation of Benefits from the Chesapeake Bay TMDLs

Abt Associates Inc. pg. 1-2

expressed with meaningful metrics. The final benefit estimates will rely on the results from a set of

water quality and ecological models that forecast conditions under various policy and baseline

scenarios. The choice of attributes to include on the survey is limited by the ability of these models to

forecast conditions under baseline and policy scenarios over the next two decades. It is the

intersection of those two sets – attributes people care about and those we are able to forecast – that

forms our choice set for attributes. The Focus Group and Interview Summary provides more details

on how each discussion informed the choice of attributes to include on the survey.

Focus groups were also used to compare two different versions of the survey. The “input”

version includes two attributes that are considered inputs into the ecological production function

(dissolved oxygen and aquatic grasses). It also includes two attributes that could be considered “dual

commodities” (Bay water clarity and lake conditions), meaning that they are ecological inputs but

people may also care about them directly. On the “endpoint” version of the survey the input

attributes are replaced by commodities that are more likely to enter individuals’ utility functions

directly (crab and oyster populations). Focus groups were used to gauge how participants related to

the attributes, made tradeoffs between them, and interpreted the changes between baseline and policy

scenarios. In the end, focus group discussions did not clearly favor one version of the survey over the

other, so the decision was made to employ both versions and compare results.

Attributes included on “input” version of the survey

Lake Conditions – Focus groups indicated that people value the improvements to freshwater lakes,

rivers, and streams. Impacts to rivers and streams would result largely from sediment reductions

since the movement of the water keeps it oxygenated even when nutrient levels are high. However,

the impacts of sediment on rivers – clarity and sedimentation of the bottom – cannot be projected with

available models. Changes in the eutrophication state of lakes, however, can be forecasted and linked

to water clarity, color and dominant fish types. Focus group respondents reacted well to the inclusion

of lake conditions in the choice scenario.

Dissolved Oxygen – One of the primary criteria used to develop the TMDL is dissolved oxygen

levels for the tidal sections of the Chesapeake Bay and its tributaries. DO criteria are developed for

142 different segments of the bay based on the type of aquatic life that should be able to live in each

segment. The DO attribute is used to describe one aspect of ecological health for the Bay on the input

version of the survey.

Subaquatic Vegetation – SAV provides habitat for many adult and juvenile Bay species including

striped bass and blue crab. It is also considered a good indicator of overall Bay health.

Water Clarity – Admittedly a dual commodity that is correlated with SAV, clarity/visibility was the

most frequently mentioned water quality characteristic in focus groups. Put simply, people want to

see their feet when they wade into the water.

Attributes included on “endpoint” version of the survey in addition to Lake Conditions and

Water Clarity

Blue Crab – Early focus groups suggested putting “all shellfish” together as an attribute. In later

discussions Maryland residents in particular thought that blue crab was an iconic species and

important to include on its own.
Peer Review Package for the Stated Preference Estimation of Benefits from the Chesapeake Bay TMDLs

Abt Associates Inc. pg. 1-3

Oysters – Identified as an important attribute because of their filtering function and for their very low

current populations. Percent changes in oyster populations are potentially large under the TMDL.

Attributes considered, but not included

Striped Bass and Recreational Fish - Bundled as an ‘endpoint’ attribute in early versions of the

survey. While this attribute was salient and can be modeled it was dropped to keep the number of

attributes on both versions the same and to reduce the cognitive burden on the endpoint survey.

Water Quality Index – A 100-point water quality index based on dissolved oxygen and clarity was

tested in early drafts of the survey. While an index could be modeled, focus group participants were

unable to interpret what specific levels and changes in the index meant.

Bird populations – While early focus groups indicated that bird populations were important, we are

not able to model changes resulting from TMDL.

Terrestrial wildlife – Not able to model changes resulting from TMDL and may not be substantially

affected by TMDL actions.

1.2.2 Attribute Metrics and Presentation

The Lake Condition attribute is defined by one of seven categories each referring to a different

range of trophic state index. The seven categories were created and linked to lake characteristics by

Carlson and Simpson (1996). A similar characterization of lake conditions was used by Phaneuf et

al. (2011) in a recent contingent valuation survey.

Dissolved Oxygen is defined by the percent of Bay area meeting the criteria developed by the

EPA to protect the aquatic life that should be present in each designated segment. Early versions

referred to the Bay segments but confused some participants.

The remaining measures are defined primarily as physical measures in absolute terms.

In focus groups respondents tended to view changes in a 100-point scale (e.g., percentage point

changes) as equal, equating a change of, for example, a 15% increase in water clarity with a 15%

increase in blue crab population. By emphasizing physical quantities such as feet of visibility, the

number of crabs, tons of oysters, and acres of SAV, respondents appeared to evaluate the changes

more carefully and not equate percentage point changes.

 Water clarity levels are expresses as “average of x feet of visibility.”

 Subaquatic Vegetation is defined using “acres of bay grass habitat.”

 Blue Crab is expressed as the number of adult crab in the Bay.

 Oyster levels are expressed in tons. When reporting current populations and setting goals for

oyster reef restoration, weight is the metric of choice for oysters.

Participants wanted the direction of the change to be clear, and the small arrows were developed

and tested in several focus groups. Additionally, some respondents wanted the percent change in the

attribute and this information is included in smaller type. Most respondents said that they found both

the absolute measure and the percentage change to be useful.
Peer Review Package for the Stated Preference Estimation of Benefits from the Chesapeake Bay TMDLs

Abt Associates Inc. pg. 1-4

1.2.3 Information Section

Development of the information section began with the fourth focus group during which it became

clear that participants wanted the following types of information:

1. A description of the problem that the policy is meant to remedy;

2. A clear description of links between pollutants and attributes (i.e. the ecological production

function);

3. Specific information on the management practices that would reduce nutrient and sediment

loads; and

4. How policy outcomes would affect them directly (i.e. changes to “endpoint” attributes).

Subsequent focus groups were used to test those findings and develop an information section that

answered those questions.

1.2.4 Payment Vehicle

There were several criteria for choosing a payment vehicle that narrowed the set of choices

considerably. But most importantly, the payment vehicle has to be believable to respondents and

applicable to households outside the watershed. The costs of meeting the TMDL will be borne by the

states of the Chesapeake Bay watershed but the sample frame includes states outside of the

watershed. For those states an increase in their utility bills or a tax would not be believable. Or

worse, it could falsely imply water quality improvements local to those respondents.

To address these restrictions the survey describes a permanent cost of living increase that, for

households within the watershed, would result from, “Higher water bills or increased maintenance

costs for septic systems for households in the watershed. For renters, rent or utility bills would

increase.” The survey also describes, “Higher prices for some agricultural products and other goods

for households both outside and inside the Watershed, including the area where you live. This is

because of higher costs for businesses inside the Watershed.”

Focus group and interview participants responded well to this payment vehicle. When

temporary taxes or fees were tested, most respondents did not believe that the cost would go to zero

when the stated duration expired. The use of taxes and fees as a payment vehicle also raised concerns

that the money would not actually be used to improve the condition of the Bay and that high

administration costs would blunt the effectiveness of the program.

1.2.5 Attribute Levels

Estimating WTP for changes in the attributes on the “input” version of the survey requires

projections of water clarity, compliance with DO criteria, acres of bay grass habitat, and the trophic

state of lakes in the watershed under the baseline and policy scenarios. The Chesapeake Bay Water

Quality Model1 was developed to determine if the States’ implementation plans would be sufficient to

meet the load reductions required under the TMDL and can be used to project the first three

1

http://www.chesapeakebay.net/about/programs/modeling
Peer Review Package for the Stated Preference Estimation of Benefits from the Chesapeake Bay TMDLs

Abt Associates Inc. pg. 1-5

attributes. Projections of trophic state (table 1) are generated by the Northeast Lakes Model2 which

was developed by EPA’s Office of Research and Development to estimate ancillary benefits to

freshwater lakes from management practices designed to improve water quality in coastal estuaries.

In addition to the two “dual commodities,” water clarity and lake condition which appear on

both versions of the survey, estimating WTP for changes in the “endpoint” attributes requires

projections of changes to blue crab and oyster populations in the Bay. The Chesapeake Bay Fisheries

Ecosystem Model (CBFEM)3is able to use projections of nutrient loadings from the Chesapeake Bay

Water Quality Model to forecast changes in relative biomass for a number of Chesapeake species,

including blue crab and oysters. Nutrient loadings under the baseline and policy scenarios produce

corresponding changes in relative biomass, which can be applied to current observed levels to

forecast changes in the attributes on the endpoint version of the survey.

1.2.6 Question Format

By varying the levels of attributes independently in the choice questions it is possible to estimate

a benefit function and shadow values for individual attributes. The contingent valuation method,

where attributes are bundled and a fixed policy scenario is compared to a fixed baseline scenario,

cannot be used to isolate the contribution of a single attribute to total WTP. The water quality and

fisheries models that produce the projections are new and are likely to be updated before a final report

is produced. To ensure we will be able to estimate benefits for a variety of policy outcomes, the

choice experiment approach was chosen over contingent valuation.

1.2.7 Baselines

The survey employs two baselines. In the “declining baseline” attribute levels worsen by 2025 in

the absence of further action; in the “constant baseline” attributes are the same in 2025 as they are

today. This is to support the TMDL benefits analysis which will employ multiple baselines,

reflecting alternative assumptions about controls already in place. We were particularly concerned

about the credibility of a constant baseline and the text was revised several times to provide adequate

support for the assertion. Key in this was to make it clear that there are controls currently in place

that would remain and to define the status quo as “No Further Action.” Additionally, early focus

groups indicated a particular concern about stalling a worsening baseline, although perceptions of

current trends varied.

1.2.8 Provision Mechanism

The survey language is intentionally non-specific about exactly which actions will be taken to

achieve improvements, but it does provide specific examples of what might be done. Specific actions

by states to meet TMDL requirements are still in development and include a wide array of

alternatives. A non-specific delivery mechanism allowed for a common treatment across the full

sample. By stating that different ways to meet the TMDL requirements produce different outcomes,

respondents were able to accept that attributes would improve at different rates, and that some

2

http://www.epa.gov/region1/neaeb2009/pdfs/20_4_Milstead_NEAEB_20090319.pdf

3

http://chesapeakebay.noaa.gov/ecosystem-modeling/chesapeake-bay-fisheries-ecosystem-model
Peer Review Package for the Stated Preference Estimation of Benefits from the Chesapeake Bay TMDLs

Abt Associates Inc. pg. 1-6

attributes may improve while others do not. The language in this section was revised several times to

respond to the need for some respondents to know more about the actual programs. The inclusion of

specific examples provided most respondents with enough specificity to answer choice questions.

1.3 Experimental Design

1.3.1 Survey Treatments

With the “input” and “endpoint” versions of the survey each presenting constant and declining

baselines, four distinct versions of the survey will be administered (table 2).

Fractional factorial design will be used to construct choice questions with an orthogonal array of

attribute levels (table 3), with questions randomly divided among distinct survey versions. Dominant

choices will be eliminated from the design since they tend to produce protest responses and provide

little useful information about preferences. The result is a 72-question experimental design when

only testing for main effects4. With 3 choice questions on each survey, 24 different treatments will be

administered for each version of the survey (A, B, C or D).

Generating forecasts of blue crab and oyster populations for the endpoint version of the

survey requires all of the modeling of the input version with an additional modeling step using

CBFEM. That and the fact that CBFEM is based on new science that contradicts the general result

that more nutrients increase biological productivity in a waterbody, and thus lead to more fish, has led

NCEE to rely primarily on the input version for the benefit analysis and to administer the endpoint

version mainly for methodological purposes. As a result, the input version of the survey will be more

widely administered than the endpoint version.

1.3.2 Sampling

NCEE will use three geographic strata to administer the survey:

1. Bay States and the District - MD, VA, DC

2. Watershed States - DE, NY, PA, and WV

3. Eastern States - VT, NH, MA, NJ, CT, RI, ME, OH, NC, SC, GA, and FL.

Figure 1 shows a map of the sampling strata and table 4 summarizes the stratification plan.

The Bay and Watershed states were chosen based on their immediate proximity to the Bay

and/or lakes, streams and rivers in its watershed (i.e.., Maryland, Virginia, the District of Columbia,

Delaware, New York, Pennsylvania, West Virginia); this sub-population of households is likely to

hold “use” values for Chesapeake Bay improvements. The remaining Eastern states (i.e., Vermont,

New Hampshire, Massachusetts, New Jersey, Connecticut, Rhode Island, Maine, Ohio, North

Carolina, South Carolina, Georgia, and Florida) are directly adjacent to the states bordering the Bay

and its watershed and represent the sub-population of households likely to hold non-use values. EPA

decided to target households in these states given their location outside the Chesapeake Bay

4 A 90-question design would be necessary if we wished to test for interaction effects between attributes.
Peer Review Package for the Stated Preference Estimation of Benefits from the Chesapeake Bay TMDLs

Abt Associates Inc. pg. 1-7

watershed, but within sufficient geographic distance that households may be familiar with

Chesapeake Bay.

The target responding sample size for the survey is 2,304 completed household surveys. This

sample size was chosen to provide statistically robust regression modeling while minimizing the cost

and burden of the survey. Given this sample size, the level of precision achieved by the analysis will

be more than adequate to meet the analytic needs of the benefits analysis for the Chesapeake Bay

TMDLs. Relative to the geographic distribution of households across the East Coast, we plan to oversample households in states adjacent to the Chesapeake Bay and within the Chesapeake Bay

watershed. We believe this approach is appropriate because households in these areas will incur the

costs of Chesapeake Bay water quality improvements, and are most likely to receive use-value

benefits from the improvements. Within each survey region, the household sample will be allocated

in proportion to the geographic distribution of households within states in the regions (Bay states,

Watershed States, East Coast states). The total number of households in each region is given below

(Table 5).

Appendices A, B, C, and D provide copies of the 4 versions of the survey instrument. Appendix E

provides a summary of the focus group results and cognitive interviews.

1.4 References

Bunch, D.S., and R.R. Batsell. 1989. “A Monte Carlo Comparison of Estimators for the Multinomial

Logit Model.” Journal of Marketing Research 26: 56-68.

Carlson, R.E. and J. Simpson. 1996. A Coordinator’s Guide to Volunteer Lake Monitoring Methods.

North American Lake Management Society. 96 pp.

Louviere, J.J., D.A. Hensher, and J.D. Swait. 2000. Stated Preference Methods: Analysis and

Application. Cambridge, UK: Cambridge University Press.

Phaneuf , D.J., von Haefen, R. H., Mansfield, C and Van Houtven, G. 201.1 Measuring Nutrient

Reduction Benefits for Policy Analysis Using Linked Non-Market Valuation and

Environmental Assessment Models: Final Report on Stated Preference Surveys.
Peer Review Package for the Stated Preference Estimation of Benefits from the Chesapeake Bay TMDLs

Abt Associates Inc. pg. 1-8

1.5 Tables and Figures

Table 1 Lake Condition Categories

Lake

Condition Water Color Water Clarity Most CommonTypes of fish

1 Green Less than 1 foot Mostly rough fish

2 Green About 1 foot Mostly rough fish

3 Brown/Green About 3 feet Bass and rough fish

4 Blue/Brown About 5 feet Mostly bass

5 Blue/Brown About 8 feet Mostly Bass and some walleye

6 Blue About 15 feet Bass, walleye, and trout

7 Blue More than 20 feet Bass, walleye, and trout

Table 2 Survey Versions

Constant Baseline Declining Baseline

Attributes are

Inputs & Dual Version A Version C

Attributes are

Endpoints &

Dual

Version B Version D
Peer Review Package for the Stated Preference Estimation of Benefits from the Chesapeake Bay TMDLs

Abt Associates Inc. pg. 1-9

Table 3 Attribute Levels

Inputs Levels

Percent Change

from current

levels

Compliance with DO criteria Declining Baseline 35% -

Constant Baseline 43% -

Policy Scenarios 43% -

85% -

100% -

Acres of Bat Grass Habitat Declining Baseline 38,000 -53%

Constant Baseline 80,000 0%

Policy Scenarios 80,000 0%

130,000 63%

165,000 106%

Endpoints

Blue Crab (millions) Declining Baseline 235 -6%

Constant Baseline 250 0%

Policy Scenarios 250 0%

280 12%

312 25%

Oysters (tons) Declining Baseline 2,800 -15%

Constant Baseline 3,300 0%

Policy Scenarios 3,300 0%

4,300 30%

5,250 59%

Dual Commodities

Water Clarity (feet) Declining Baseline 1.5 -50%

Constant Baseline 3 0%

Policy Scenarios 3 0%

4 33%

5 67%

Lake Condition Declining Baseline 2 -

Constant Baseline 3 -

Policy Scenarios 3 -

4 -

5 -
Peer Review Package for the Stated Preference Estimation of Benefits from the Chesapeake Bay TMDLs

Abt Associates Inc. pg. 1-10

Table 4 Sampling Plan

Geographic Divison Baseline Factor Attribute Type

Cell 1 Bay States Constant Input

Cell 2 Bay States Declining Input

Cell 3 Bay States Declining Endpoint

Cell 4 Watershed Constant Input

Cell 5 Watershed Declining Input

Cell 6 Other East Coast Constant Input

Cell 7 Other East Coast Declining Input

Cell 8 Watershed Constant Endpoint

Table 5 Number of Households and Household Sample for Each Study Region

Sampling Stratum and State Total Households Expected household

sample size

Bay States Stratum, total 5,479,176 864

District of Columbia 266,707

Maryland 2,156,411

Virginia 3,056,058

Watershed States Stratum, total 13,442,787 864

Delaware 342,297

New York 7,317,755

Pennsylvania 5,018,904

West Virginia 763,831

Other East Coast States Stratum, total 30,034,893 576

Connecticut 1,371,087

Florida 7,420,802

Georgia 3,585,584

Maine 557,219

Massachusetts 2,547,075

New Jersey 3,214,360

New Hampshire 518,973

North Carolina 3,745,155

Ohio 4,603,435

Rhode Island 413,600

South Carolina 1,801,181

Vermont 256,442

Source: U.S. Census Bureau (2012). 2010 Census Summary File 1. Retrieved May 31, 2012 from

http://factfinder2.census.gov/.
Peer Review Package for the Stated Preference Estimation of Benefits from the Chesapeake Bay TMDLs

Abt Associates Inc. pg. 1-11

Figure 1 Sampling Strata
Peer Review Package for the Stated Preference Estimation of Benefits from the Chesapeake Bay TMDLs

Abt Associates Inc. pg. 2-12

2. Charge Questions

The reviewers are encouraged to refer to past studies/efforts when answering these charge questions.

2.1 General Impression of Survey Instrument and Proposed Study

Please provide comments on the following topics:

1. Extent to which the survey results are likely to provide welfare estimates of improving water

quality in the Chesapeake Bay. Specifically,

a. the ability of the survey to provide total values

b. appropriateness of the proposed choice experiment survey for the estimation of marginal

willingness to pay for individual attributes

c. appropriateness of a choice experiment survey format for the elicitation of preferences in this

case (i.e., compared to alternatives such as contingent valuation).

2. Overall quality and design of the proposed survey.

2.2 Survey Instrument

Please comment on the following:

3. Adequacy of background information to enable the respondents to answer the choice questions. Is

the information necessary, sufficient, and accurate?

4. Does the survey provide appropriate and sufficient introductory and debriefing questions to, for

example, identify protest responses and symbolic responses, test retention and understanding of

information, elicit non-use values, etc.? Are there any additional introductory or debriefing

questions you feel may be useful for the analysis?

5. Does the survey place sufficient and appropriate emphasis on the budget constraint facing

respondents, and on potential substitute uses of their budget?

6. Considering all four versions of the survey, are the program options under consideration

appropriately and sufficiently characterized? For example, given the focus group summary

provided, are the appropriate attributes included in the choice experiment?

7. Regarding the “Input” version of the survey, are the links between attributes in the program

options (such as Dissolved Oxygen) and aspects of the Bay respondents may directly care about

appropriately described? Are there ways to improve the description of those links?

8. Does the survey incorporate appropriate language to ameliorate symbolic responses and 'yea

saying'?

9. Does the survey take appropriate steps to minimize hypothetical and strategic bias, including

sufficient steps to emphasize consequentiality?

10. Is the format of the choice questions appropriate according to current standards in the stated

preference literature (e.g., allowing for a status quo option, encouraging neoclassical tradeoffs,

order and presentation of attributes, etc.)

11.Is there evidence of fundamental or critical flaws in the survey? Such as bias in the survey or
Peer Review Package for the Stated Preference Estimation of Benefits from the Chesapeake Bay TMDLs

Abt Associates Inc. pg. 2-13

presented information?

2.3 Study Design

12.Are the proposed attribute levels and use of a constant versus declining baseline appropriate for

estimating the benefits of the TMDL?

13.Is the sampling plan and proposed geographic stratification appropriate for estimating the total

benefits of the TMDL, including nonuse values?
Attachment A. Stated Preference Questionnaire, Attributes are Inputs

& Dual, Constant Baseline
Chesapeake Bay Stated Preference

The Future of the Chesapeake Bay: 

What’s Your Opinion?
About this survey

We are conducting this survey about the Chesapeake Bay Watershed (that is 

both the Bay and the lands around it). The survey’s purpose is to collect 

information to help policymakers make decisions that affect water quality, wildlife 

and recreation. 

In this survey, the first 

questions are about: 

 Your experiences with the 

Chesapeake Bay Watershed

 Your thoughts about the 

Watershed’s current 

conditions

Then the survey describes: 

 Different plans that could 

affect the Watershed

 What changes might result 

from each plan

 How a plan might affect 

households like yours 

We want to know which of these plans you would vote for. Finally we ask a few 

questions about your opinions on the survey’s topics, and some background 

information about your household.

Your responses to this survey will help to inform policymakers how the public feels 

about plans that would affect the Chesapeake Bay Watershed and your household. 

This is your opportunity to voice your opinions about these plans. Please 

take the time to read the information in this survey and answer the questions 

carefully.

All information and responses will be kept confidential. Background information in 

this survey was provided by the National Marine Fisheries Service, U.S. 

Environmental Protection Agency, and other state and federal offices.

1
The Chesapeake Bay Watershed

The Chesapeake Bay Watershed is shown below in green. 

The Watershed spreads over parts of six states and Washington DC. It contains 150 

major rivers and thousands of lakes, small rivers and streams. Water draining 

from these lands enters these waterways and eventually the Chesapeake Bay. 

The Chesapeake Bay Watershed includes two types of water bodies, with different 

characteristics and potential problems.

2

Tidal waters include the Bay 

itself and parts of some rivers and streams 

that flow into it. Tidal waters are salty and 

support different types of wildlife than 

freshwater, but are also used to swim, 

boat, and fish.

Freshwater lakes and 

rivers support wildlife and 

are used for swimming, 

boating, and fishing, among 

other activities.
3

Lakes and rivers 

in the Watershed

Chesapeake Bay 

tidal waters

Swimming, water skiing, or 

jet skiing

□ □

Fishing □ □

Boating, including sailing, 

motor boating and kayaking

□ □

Viewing nature, including 

bird watching □ □

Biking or walking on trails 

along the water □ □

Other 

_______________________

□ □

1. Over the last five years, which of these activities have you done 

in any freshwater lakes or rivers, or in the Bay’s tidal waters?
2. In the last five years, about how often have you visited 

watershed lakes or rivers, or the Chesapeake Bay Tidal waters 

to do any of these activities? 

In each column please check the statement that best describes 

your experience.

4

Lakes and rivers 

in the Watershed

Chesapeake Bay 

tidal waters

I have never visited □ □

I have not visited in the last 

5 years □ □

Once over the last 5 years □ □

A few times over the last 5 

years 

□ □

About once a year □ □

2 to 10 times a year □ □

More than 10 times a year □ □
5

Water Quality in the 

Chesapeake Bay Watershed

For the following questions, check the box next to your answer.

3. Before taking this survey, were you aware that too much 

sediment or nutrients can harm wildlife habitats?

Yes

No

4. Have you previously heard of any sources of excess nutrients and 

sediments in the Chesapeake Bay Watershed? 

Yes

No

Pollutants in the Chesapeake Bay Watershed degrade the quality of the 

water, and can affect wildlife habitat and recreational opportunities. Two key 

pollutants are nutrients and sediments.

 Nutrients, such as nitrogen and phosphorous occur naturally in the 

watershed, and are essential for healthy ecosystems. But excess nutrients 

(from fertilizers, livestock manure, and wastewater treatment plants) can lead 

to algae that deprives fish and plants of oxygen.

 Sediment. Some loose soil naturally settles to the bottom of water bodies. 

Too much sediment makes the water murky and harms aquatic plants and 

animals. Paved surfaces and some farming practices increase soil erosion, 

causing too much sediment to enter rivers and streams that flow into the Bay.
6

Reducing excess nutrients and sediment in 

the Chesapeake Bay Watershed

Changing the way land, storm water, and waste water are managed can improve 

water quality in the Chesapeake Bay Watershed. As a result, wildlife habitat and 

recreation opportunities will improve in lakes, rivers and the Bay.

Practices to Improve Water Quality

 Changing the way farmers dispose of livestock manure and till their land 

 Installing better equipment at wastewater treatment facilities

 Restoring streams and wetlands and paving fewer surfaces to slow storm 

water runoff 

 Restoring oyster reefs and bay grass beds to filter water and improve 

fish habitat

Water Quality Improvements

 Clearer water because less nutrients and sediment enter the water 

 More aquatic grasses will grow because clearer water allows more sunlight to 

reach them

 Smaller, less frequent algae blooms 

 More oxygen in the water because of reduced algae 

Ecosystem and Recreation Improvements

 Cleaner and healthier aquatic environment 

 More game fish and crabs for recreational fishermen

 More enjoyable boating and swimming 

 More oysters and bay grass beds will provide habitat for a variety of fish and 

shellfish while also improving water quality
5. How do you think environmental conditions have changed 

in lakes and rivers closest to your home and in the 

Chesapeake Bay? (Please check one box in each column.)

7

Lakes and rivers Chesapeake Bay

Better than they were 10 years 

ago 

□ □

About the same as they were 

10 years ago □ □

Worse than they were 10 years 

ago □ □

Don’t know □ □

Strongly

Disagree

Neutral Strongly 

Agree

Don’t 

Know

I am concerned about current 

pollution levels in lakes and rivers. 1 2 3 4 5 DK

I am concerned about current 

pollution levels in the Chesapeake 

Bay.

1 2 3 4 5 DK

Some decrease in environmental 

quality is inevitable as our economy 

grows.

1 2 3 4 5 DK

6. On a scale from 1 to 5, please circle the number that indicates how 

much you agree or disagree with each statement. 
8

Environmental Quality in the 

Chesapeake Bay

For the Chesapeake Bay, environmental goals are based on several 

measurements of environmental quality and aquatic life:

Dissolved Oxygen

 Dissolved oxygen refers to oxygen in the water, something all aquatic life 

needs

 Dissolved oxygen goals were set for different areas of the Chesapeake Bay 

based on the types of aquatic life that live there

Water Clarity

 Water clarity is measured by how far one can see into the water

 Clearer water means more sunlight for underwater plants to grow

 People who go swimming or boating may prefer clearer water

Amount of Aquatic Grass Habitat

 Aquatic grasses provide food and habitat for other aquatic life 

Different areas of the Chesapeake Bay provide different types of wildlife habitat and 

recreation opportunities. 

State governments and federal agencies have been working with scientists to develop 

environmental goals for water quality in the Chesapeake Bay.

7. Before taking this survey, had you read or heard about any of 

these environmental goals for dissolved oxygen, water clarity, or 

aquatic grasses? 

Yes

No
9

Environmental Quality in the 

Chesapeake Bay

State and local governments already have some pollution reduction programs in place 

but some additional actions are being considered. Scientists studying the Chesapeake 

Bay Watershed monitor its health and predict changes in water quality under different 

pollution reduction programs. 

The table here shows both current conditions and predicted conditions in 2025

for the Chesapeake Bay tidal waters if no further action is taken to reduce nutrients 

and sediment.

Current Conditions Conditions in 2025 with 

no further action *

Dissolved Oxygen (DO)

Percent of waters meeting 

DO goals

Some plans being 

considered would achieve 

100% in 2025

40 %

of the Bay areas

meet DO goals 

40 %

of the Bay areas

will meet DO goals

Water Clarity 

How many feet you can see 

down into the water.

Average visibility of

3 feet

Average visibility of

3 feet

no change

Aquatic Grasses 

Acres of underwater grasses 

on the Bay floor.

Scientists recommend 

185,000 acres be restored 

based on historic levels 

80,000 acres 80,000 acres

no change

*All forecasts for the year 2025 are based on monitoring data from the Chesapeake Bay Watershed 

and Estuary Models developed by the Chesapeake Bay Program Office of the EPA in conjunction with 

state and federal partners.
10

Environmental Quality in the Lakes

Scientists studying the effect of nutrient pollution on lakes categorize them by how 

much algae grows in the water.

High amounts of algae:

 Changes the color of the water and reduces visibility. 

 Results in fewer game fish like bass and trout and more rough fish like carp and 

catfish.

Lake 

Condition

Water 

Color

Water 

Clarity 

Most Common Types 

of Fish

1 Green Less than 1 foot Mostly rough fish

2 Green About 1 foot Mostly rough fish

3 Brown/Green About 3 feet Bass and rough fish

4 Blue/Brown About 5 feet Mostly bass

5 Blue/Brown About 8 feet Mostly bass and some 

walleye

6 Blue About 15 feet Bass, walleye, and trout

7 Blue More than 20 feet Bass, walleye, and trout

8. What condition do you think best describes the lakes closest to your 

home? (Please circle one.) 

1 2 3 4 5 6 7

Don’t 

Know
11

Lakes in the Chesapeake Bay Watershed

Some important things to note: 

 Meeting the goals for water quality in the Chesapeake Bay would also affect 

freshwater bodies of the watershed. 

 Nutrients have a small effect on many rivers because the water is constantly 

moving. 

 Reducing the amount of nutrients entering lakes will improve the appearance of 

the water and change the ecological conditions. 

The table below shows current conditions and conditions in 2025 that scientists 

predict for lakes in the part of the watershed in your state if no further actions are 

taken to reduce nutrient and sediment pollution.

The table shows the average condition for <state> lakes within the Watershed. This is 

an average, so individual lakes near you could be higher or lower. 

Current Conditions Conditions in 2025 with

no additional actions*

Average Condition 

of <State> Lakes in 

the Watershed

3

Brown/Green Water

3 feet of visibility

Bass and rough fish 

3

Brown/Green Water

3 feet of visibility

Bass and rough fish 

*All forecasts for the year 2025 are based on measures from the Northeast Lakes Model developed by 

the Office of Research and Development of the EPA.
12

Pollution Reduction Programs for the 

Chesapeake Bay Watershed

9. Before taking this survey, had you read or heard of programs to 

reduce excess nutrients and sediments in the Chesapeake Bay?

Yes

No

Progress toward each water quality goal will depend on the design of 

the program.

For example,

 Pollution reduction practices close to the Bay will improve water quality in 

the Bay, but not in freshwater lakes farther out in the watershed.

 Planting aquatic grasses will improve aquatic life habitat but will have a 

smaller effect on water clarity and dissolved oxygen than programs focused 

on reducing nutrients and sediment.

Federal and State Agencies are developing programs to meet the goals that have been 

set for water quality and wildlife in the Chesapeake Bay. Any program would be phased 

in over time but would be fully implemented by the year 2025.
13

Paying for Pollution Reduction Programs

Some of the basic things people spend money on would become more expensive. 

For example:

 Higher water bills or increased maintenance costs for septic systems for 

households in the Watershed. For renters, rent or utility bills would increase.

 Higher prices for some agricultural products and other goods for households 

both outside and inside the Watershed, including the area where you live. 

This is because of higher costs for businesses inside the Watershed.

A pollution reduction program, if implemented, would permanently increase the 

cost of living for your household.

The changes required under pollution reduction programs would result in higher costs 

for your household. 
14

Deciding Future Actions

Imagine that you were given the opportunity to vote on future pollution reduction 

programs. 

In the questions that follow, we will ask your opinion about different programs that 

provide different levels of environmental improvements, and cost your household 

different amounts.

You will be asked three questions. In each question you will need to choose the 

program you like best from the following alternatives: 

 The NO FURTHER ACTION option would have no additional pollution reductions 

or costs to your household.

 The two programs labeled PROGRAM A and PROGRAM B differ in how much they 

reduce pollution and how much they cost your household.

An Example Question is on the next page to show you what the questions will 

look like.
15

An Example Question

Conditions in 2025

Current 

Conditions

No Further 

Action Program A Program B

Dissolved 

Oxygen

40% of the Bay

areas will meet DO 

goals 

40%

of the Bay areas

will meet DO goals 

50%

of the Bay areas

will meet DO goals

70%

of the Bay areas

will meet DO goals

Water Clarity

Average visibility 

of 3 feet 

Average visibility of

3 feet

no change

Average visibility of

4 feet

33% increase 

Average visibility of

3 feet

67% increase 

Aquatic 

Grasses

Coverage is 

80,000 acres

80,000 acres

no change

100,000 acres

25% increase 

150,000 acres

50% increase 

Lake Condition

Average of 3

in <state> 

3

Brown/Green Water

3 feet of visibility

Bass and rough fish 

4

Blue/Brown Water

5 feet of visibility

Mostly Bass

3

Brown/Green Water

3 feet of visibility

Bass and rough fish 

Permanent 

Increase in the 

Cost of Living for 

your Household 

Your Vote

Please mark one of 

the boxes to the 

right 

$0 per year 

No Further Action 

$36 per year

or $3 per month 

Program A 

$50 per year

or $4.16 per month

Program B 

Chesapeake Bay and Tidal Waters Lakes

If you do not want A 

or B, mark this box

If you want Program A, 

mark this box

If you want Program B, 

mark this box
16

Consider the Outcomes and the Costs

Please take time to consider both the benefits of the program and the costs to 

your household. Ask yourself if the program outcomes for the Chesapeake Bay 

Watershed are worth the additional cost to your household. 

Paying the costs means you would have less money to spend on other things 

such as food, clothes, going on trips, and even towards resolving other 

environmental problems you care about. 

You would be making a commitment to pay the additional amount every year 

from now on. 

Only you know what is best for you and your household, so please only think of your 

household when deciding whether to vote for NO FURTHER ACTION, PROGRAM A, or 

PROGRAM B.

Please think carefully about how you would actually vote in these situations. We urge 

you to respond as though costs for your household really would go up if the program 

were implemented. Knowing how different people would vote on this program is very 

important for decision makers in state and federal governments.

Weighing Outcomes and Costs
17

Conditions in 2025

Current 

Conditions

No Further 

Action Program A Program B

Dissolved 

Oxygen

40% of the Bay

areas will meet DO 

goals 

40%

of the Bay areas

will meet DO goals 

85%

of the Bay areas

will meet DO goals

100%

of the Bay areas

will meet DO goals

Water Clarity

Average visibility 

of 3 feet 

Average visibility of

3 feet

no change

Average visibility of

5 feet

83% increase 

Average visibility of

4 feet

50% increase 

Aquatic 

Grasses

Coverage is 

80,000 acres

80,000 acres

no change

130,000 acres

25% increase 

165,000 acres

50% increase 

Lake Condition

Average of 3

in <state> 

3

Brown/Green Water

3 feet of visibility

Bass and rough fish 

5

Blue/Brown Water

8 feet of visibility

Bass and walleye

4

Blue/Brown Water

5 feet of visibility

Mostly bass 

Permanent 

Increase in the 

Cost of Living for 

your Household 

Your Vote

Please mark one of 

the boxes to the 

right 

$0 per year 

No Further Action 

$250 per year

or $20.83 per 

month 

Program A 

$180 per year

or $15 per month

Program B 

Chesapeake Bay and Tidal Waters Lakes

10. Suppose you could choose from the following three options. Mark

 the box at the bottom to indicate which plan you would vote for.
18

As you vote on the next questions, please remember...

 On the following pages you will see questions that present new sets of program 

options. These options require a different mix of management practices in different 

areas.

 Consider each question separately, imagining that the options are the only ones 

available to choose from.

 The next two questions cannot be compared to each other, or to the question you 

just answered.

 Do not add up effects or costs across different questions. 
19

Conditions in 2025

Current 

Conditions

No Further 

Action Program A Program B

Dissolved 

Oxygen

40% of the Bay

areas will meet DO 

goals 

40%

of the Bay areas

will meet DO goals 

85%

of the Bay areas

will meet DO goals

40%

of the Bay areas

will meet DO goals

Water Clarity

Average visibility 

of 3 feet 

Average visibility of

3 feet

no change

Average visibility of

4 feet

33% increase 

Average visibility of

5 feet

no change 

Aquatic 

Grasses

Coverage is 

80,000 acres

80,000 acres

no change

130,000 acres

25% increase 

80,000 acres

50% increase 

Lake Condition

Average of 3

in <state> 

3

Brown/Green Water

3 feet of visibility

Bass and rough fish 

3

Brown/Green Water

3 feet of visibility

Bass and rough fish

4

Blue/Brown Water

5 feet of visibility

Mostly bass 

Permanent 

Increase in the 

Cost of Living for 

your Household 

Your Vote

Please mark one of 

the boxes to the 

right 

$0 per year 

No Further Action 

$40 per year

or $3.33 per month 

Program A 

$60 per year

or $5 per month

Program B 

Chesapeake Bay and Tidal Waters Lakes

11. Now suppose instead that a different set of options were available. 

 Please mark the box at the bottom to indicate which plan you 

 would vote for.
20

Conditions in 2025

Current 

Conditions

No Further 

Action Program A Program B

Dissolved 

Oxygen

40% of the Bay

areas will meet DO 

goals 

40%

of the Bay areas

will meet DO goals 

100%

of the Bay areas

will meet DO goals

85%

of the Bay areas

will meet DO goals

Water Clarity

Average visibility 

of 3 feet 

Average visibility of

3 feet

no change

Average visibility of

5 feet

83% increase 

Average visibility of

4 feet

33% increase 

Aquatic 

Grasses

Coverage is 

80,000 acres

80,000 acres

no change

130,000 acres

20% increase 

165,000 acres

75% increase 

Lake Condition

Average of 3

in <state> 

3

Brown/Green Water

3 feet of visibility

Bass and rough fish 

5

Blue/Brown Water

8 feet of visibility

Bass and walleye

4

Blue/Brown Water

5 feet of visibility

Mostly bass 

Permanent 

Increase in the 

Cost of Living for 

your Household 

Your Vote

Please mark one of 

the boxes to the 

right 

$0 per year 

No Further Action 

$370 per year

or $30.83 per 

month 

Program A 

$180 per year

or $15 per month

Program B 

Chesapeake Bay and Tidal Waters Lakes

12. Now suppose instead that a different set of options were available. 

 Please mark the box at the bottom to indicate which plan you 

 would vote for.
21

13. For the choices you just made about programs, how much 

 consideration did you give to improvements in lakes compared 

 to the Chesapeake Bay tidal waters? (Please circle one.)

Considered 

Only the 

Lakes

Considered 

Lakes and 

Bay 

Waters 

Equally

Considered 

Only the 

Bay 

Waters

1 2 3 4 5 6 7

14. On a scale from 1 to 5, please rank the following based in order

 of importance when you were choosing between programs. 

Not 

Important

Somewhat 

Important

Very 

Important

Don’t 

Know

Dissolved oxygen 1 2 3 4 5 DK

Water clarity 1 2 3 4 5 DK

Aquatic grass coverage 1 2 3 4 5 DK

Lake category 1 2 3 4 5 DK

Cost of the program 1 2 3 4 5 DK
22

15. Aside from costs, which of the following was most important

 in making your decision about which program to vote for? 

 (Please choose one.)

Recreation on the Bay

Recreation in Lakes

Preserving the environment for future generations

Availability and/or price of seafood

Value of property near the Bay

Value of property near Lakes

The possibility of visiting the Bay in the future

The possibility of visiting Lakes in the future

Other __________________
23

16. Thinking about how you voted among programs, please rate

 how much you agree or disagree with each of the following 

 statements. (Circle the number that matches your answer.)

Strongly 

Disagree Neutral Strongly Agree

My choices would have been 

different if the economy in my 

area were better. 1 2 3 4 5 6 7

It is important to improve waters 

in the Chesapeake Bay 

watershed, no matter how high 

the costs.

1 2 3 4 5 6 7

I do not think I should have to 

contribute any amount to the 

restoration of the Chesapeake 

Bay watershed.

1 2 3 4 5 6 7

I am concerned that the 

programs would hurt the 

economy.

1 2 3 4 5 6 7

It was difficult to understand the 

descriptions of the programs. 1 2 3 4 5 6 7

I believe the programs will 

improve the environment as 

described.

1 2 3 4 5 6 7

Some of the programs cost too 

much compared to what they 

would deliver. 1 2 3 4 5 6 7

The changes offered by the 

programs happen too far into the 

future for me to really care 

about.

1 2 3 4 5 6 7

The survey provided me with the 

information I needed to make a 

choice between the options 

shown.

1 2 3 4 5 6 7
24

17. If you voted for NO FURTHER ACTION in any of the choices you

 were given, please let us know how much you agree or disagree

 with each of the following statements. If not, skip to the next 

 question. (Circle the number that matches your answer.)

Strongly 

Disagree Neutral Strongly Agree

Nothing can be done to 

improve the quality of the Bay 

and freshwater lakes.

1 2 3 4 5 6 7

I would not vote for any 

program even if there were no 

costs to my household.

1 2 3 4 5 6 7

I voted for NO FURTHER 

ACTION because the costs were 

too high compared to the 

improvements.

1 2 3 4 5 6 7

I voted for NO FURTHER 

ACTION because I am against 

more government requirements 

and spending.

1 2 3 4 5 6 7
25

Strongly 

Disagree Neutral Strongly Agree

I voted for a program because I 

thought it would also lead to 

better water quality closer to 

my home. 

1 2 3 4 5 6 7

I voted for a program more for 

future generations than for 

myself. 

1 2 3 4 5 6 7

I voted for the program 

because I may visit the 

Chesapeake Bay or lakes in the 

watershed in the next few 

years. 

1 2 3 4 5 6 7

I voted for a program because I 

believe it is our responsibility to 

preserve wildlife and habitat in 

the Bay and Lakes. 

1 2 3 4 5 6 7

I voted for a program so others 

could enjoy the Bay and lakes, 

rather than for myself. 

1 2 3 4 5 6 7

18. If you voted for Program A or Program B in any of the choices

 you were given, please let us know how much you agree or 

 disagree with each of the following statements. If not, skip this 

 question. (Circle the number that matches your answer.)
26

19. Are you a male or female?

Questions about you and your household

Finally, we would like to ask you a few questions about you and your household for 

statistical purposes. The reports prepared for this study will summarize findings 

across all responses and will not associate responses with an individual. Your 

answers will not be saved or stored in a way that can be associated with your name 

or address. You will not be contacted about your responses or this survey. 

Male

Female

20. What is your age?

______ years old

21. What is your current marital status?

Under $25,000

$25,000-$34,999

$35,000-$49,999

$50,000-$74,999

$75,000-$99,999

$100,000-$199,999

$200,000 or more

22. How many children under age 18 are living at your home?

______ children

23. In 2011, what was your total pre-tax household income, including

 all earners in your household?

Single, never married

Married or living with a long-term partner

Separated or divorced

Widowed
27

25. Which of the following best describes the home or apartment you live in?

Owned by you or someone in your household with a mortgage or loan

Owned by you or someone in your household without a mortgage or loan

Rented

Other: _____________________________________________

24. What is the highest degree or level of school you have completed?

No high school diploma

High school diploma or GED

Some college credit or college degree

Some graduate school or professional school credit or a graduate

or professional degree

26. Have you or any member of your family ever worked for any of the

 following industries or jobs? (Please fill in the box next to all that apply.)

Agriculture

Commercial fishing

Tour guide for fishing

No one in my family has worked in these industries

Thank you very much 

for your help.
The public reporting and recordkeeping burden for this collection of information is estimated 

to average 2 hours per response. Send comments on the Agency's need for this 

information, the accuracy of the provided burden estimates, and any suggested methods for 

minimizing respondent burden, including through the use of automated collection techniques 

to the Director, Collection Strategies Division, U.S. Environmental Protection Agency 

(2822T), 1200 Pennsylvania Ave., NW, Washington, D.C. 20460. Include the OMB control 

number in any correspondence. Do not send the materials to this address.
Attachment B. Stated Preference Questionnaire, Attributes are

Endpoints & Dual, Constant Baseline
Chesapeake Bay Stated Preference

The Future of the Chesapeake Bay: 

What’s Your Opinion?
About this survey

We are conducting this survey about the Chesapeake Bay Watershed (that is 

both the Bay and the lands around it). The survey’s purpose is to collect 

information to help policymakers make decisions that affect water quality, wildlife 

and recreation. 

In this survey, the first 

questions are about: 

 Your experiences with the 

Chesapeake Bay Watershed

 Your thoughts about the 

Watershed’s current 

conditions

Then the survey describes: 

 Different plans that could 

affect the Watershed

 What changes might result 

from each plan

 How a plan might affect 

households like yours 

We want to know which of these plans you would vote for. Finally we ask a few 

questions about your opinions on the survey’s topics, and some background 

information about your household.

Your responses to this survey will help to inform policymakers how the public feels 

about plans that would affect the Chesapeake Bay Watershed and your household. 

This is your opportunity to voice your opinions about these plans. Please 

take the time to read the information in this survey and answer the questions 

carefully.

All information and responses will be kept confidential. Background information in 

this survey was provided by the National Marine Fisheries Service, U.S. 

Environmental Protection Agency, and other state and federal offices.

1
The Chesapeake Bay Watershed

The Chesapeake Bay Watershed is shown below in green. 

The Watershed spreads over parts of six states and Washington DC. It contains 150 

major rivers and thousands of lakes, small rivers and streams. Water draining 

from these lands enters these waterways and eventually the Chesapeake Bay. 

The Chesapeake Bay Watershed includes two types of water bodies, with different 

characteristics and potential problems.

2

Tidal waters include the Bay 

itself and parts of some rivers and streams 

that flow into it. Tidal waters are salty and 

support different types of wildlife than 

freshwater, but are also used to swim, 

boat, and fish.

Freshwater lakes and 

rivers support wildlife and 

are used for swimming, 

boating, and fishing, among 

other activities.
3

Lakes and rivers 

in the Watershed

Chesapeake Bay 

tidal waters

Swimming, water skiing, or 

jet skiing

□ □

Fishing □ □

Boating, including sailing, 

motor boating and kayaking

□ □

Viewing nature, including 

bird watching □ □

Biking or walking on trails 

along the water □ □

Other 

_______________________

□ □

1. Over the last five years, which of these activities have you done 

in any freshwater lakes or rivers, or in the Bay’s tidal waters?
2. In the last five years, about how often have you visited 

watershed lakes or rivers, or the Chesapeake Bay Tidal waters 

to do any of these activities? 

In each column please check the statement that best describes 

your experience.

4

Lakes and rivers 

in the Watershed

Chesapeake Bay 

tidal waters

I have never visited □ □

I have not visited in the last 

5 years □ □

Once over the last 5 years □ □

A few times over the last 5 

years 

□ □

About once a year □ □

2 to 10 times a year □ □

More than 10 times a year □ □
5

Water Quality in the 

Chesapeake Bay Watershed

For the following questions, check the box next to your answer.

3. Before taking this survey, were you aware that too much 

sediment or nutrients can harm wildlife habitats?

Yes

No

4. Have you previously heard of any sources of excess nutrients and 

sediments in the Chesapeake Bay Watershed? 

Yes

No

Pollutants in the Chesapeake Bay Watershed degrade the quality of the 

water, and can affect wildlife habitat and recreational opportunities. Two key 

pollutants are nutrients and sediments.

 Nutrients, such as nitrogen and phosphorous occur naturally in the 

watershed, and are essential for healthy ecosystems. But excess nutrients 

(from fertilizers, livestock manure, and wastewater treatment plants) can lead 

to algae that deprives fish and plants of oxygen.

 Sediment. Some loose soil naturally settles to the bottom of water bodies. 

Too much sediment makes the water murky and harms aquatic plants and 

animals. Paved surfaces and some farming practices increase soil erosion, 

causing too much sediment to enter rivers and streams that flow into the Bay.
6

Reducing excess nutrients and sediment in 

the Chesapeake Bay Watershed

Changing the way land, storm water, and waste water are managed can improve 

water quality in the Chesapeake Bay Watershed. As a result, wildlife habitat and 

recreation opportunities will improve in lakes, rivers and the Bay.

Practices to Improve Water Quality

 Changing the way farmers dispose of livestock manure and till their land 

 Installing better equipment at wastewater treatment facilities

 Restoring streams and wetlands and paving fewer surfaces to slow storm 

water runoff 

 Restoring oyster reefs and bay grass beds to filter water and improve 

fish habitat

Water Quality Improvements

 Clearer water because less nutrients and sediment enter the water 

 More aquatic grasses will grow because clearer water allows more sunlight to 

reach them

 Smaller, less frequent algae blooms 

 More oxygen in the water because of reduced algae 

Ecosystem and Recreation Improvements

 Cleaner and healthier aquatic environment 

 More game fish and crabs for recreational fishermen

 More enjoyable boating and swimming 

 More oysters and bay grass beds will provide habitat for a variety of fish and 

shellfish while also improving water quality
5. How do you think environmental conditions have changed 

in lakes and rivers closest to your home and in the 

Chesapeake Bay? (Please check one box in each column.)

7

Lakes and rivers Chesapeake Bay

Better than they were 10 years 

ago 

□ □

About the same as they were 

10 years ago □ □

Worse than they were 10 years 

ago □ □

Don’t know □ □

Strongly

Disagree

Neutral Strongly 

Agree

Don’t 

Know

I am concerned about current 

pollution levels in lakes and rivers. 1 2 3 4 5 DK

I am concerned about current 

pollution levels in the Chesapeake 

Bay.

1 2 3 4 5 DK

Some decrease in environmental 

quality is inevitable as our economy 

grows.

1 2 3 4 5 DK

6. On a scale from 1 to 5, please circle the number that indicates how 

much you agree or disagree with each statement. 
8

Environmental Quality in the 

Chesapeake Bay

For the Chesapeake Bay, environmental goals are based on several 

measurements of water quality and aquatic life:

Water Clarity

 Measured by how far one can see into the water

 People who go swimming or boating my prefer clearer water

 Clearer water means more sunlight for underwater plants to grow

Blue Crab

 A key ecological species and popular shellfish for recreational fishermen

Oysters

 Their shells form reefs that provide habitat for other aquatic life

 As “filter feeders” they clean the waters of the Bay 

Different areas of the Chesapeake Bay provide different types of wildlife habitat and 

recreation opportunities. 

State governments and federal agencies have been working with scientists to develop 

environmental goals for improving the quality of the Chesapeake Bay.

7. Before taking this survey, had you read or heard about any of 

these environmental goals for water clarity in the Chesapeake Bay

or for any Chesapeake Bay species? 

Yes

No
9

Environmental Quality in the 

Chesapeake Bay

State and local governments already have some pollution reduction programs in place 

but some additional actions are being considered. Scientists studying the Chesapeake 

Bay Watershed monitor its health and predict changes in water quality under different 

pollution reduction programs. 

The table here shows both current conditions and predicted conditions in 2025

for the Chesapeake Bay tidal waters if no further action is taken to reduce nutrients 

and sediment.

Current Conditions Conditions in 2025 with 

no further action *

Water Clarity

How many feet you can see 

down into the water

Average visibility of

3 feet

Average visibility of

3 feet

no change

Blue Crabs 

Goal is 415 million crabs

250

million crabs

250

million crabs

no change 

Oysters 

Goal is 35,000 tons

3,300

tons

3,300

tons

no change

*All forecasts for the year 2025 are based on monitoring data from the Chesapeake Bay Watershed 

and Estuary Models developed by the Chesapeake Bay Program Office of the EPA in conjunction with 

state and federal partners.
10

Environmental Quality in the Lakes

Scientists studying the effect of nutrient pollution on lakes categorize them by how 

much algae grows in the water.

High amounts of algae:

 Changes the color of the water and reduces visibility. 

 Results in fewer game fish like bass and trout and more rough fish like carp and 

catfish.

Lake 

Condition

Water 

Color

Water 

Clarity 

Most Common Types 

of Fish

1 Green Less than 1 foot Mostly rough fish

2 Green About 1 foot Mostly rough fish

3 Brown/Green About 3 feet Bass and rough fish

4 Blue/Brown About 5 feet Mostly bass

5 Blue/Brown About 8 feet Mostly bass and some 

walleye

6 Blue About 15 feet Bass, walleye, and trout

7 Blue More than 20 feet Bass, walleye, and trout

8. What condition do you think best describes the lakes closest to your 

home? (please circle one.) 

1 2 3 4 5 6 7

Don’t 

Know
11

Lakes in the Chesapeake Bay Watershed

Some important things to note: 

 Meeting the goals for water quality in the Chesapeake Bay would also affect 

freshwater bodies of the watershed. 

 Nutrients have a small effect on many rivers because the water is constantly 

moving. 

 Reducing the amount of nutrients entering lakes will improve the appearance of 

the water and change the ecological conditions. 

The table below shows current conditions and conditions in 2025 that scientists 

predict for lakes in the part of the watershed in your state if no further actions are 

taken to reduce nutrient and sediment pollution.

The table shows the average condition for <state> lakes within the Watershed. This is 

an average, so individual lakes near you could be higher or lower. 

Current Conditions Conditions in 2025 with

no additional actions*

Average Condition 

of <State> Lakes in 

the Watershed

3

Brown/Green Water

3 feet of visibility

Bass and rough fish 

3

Brown/Green Water

3 feet of visibility

Bass and rough fish 

*All forecasts for the year 2025 are based on measures from the Northeast Lakes Model developed by 

the Office of Research and Development of the EPA.
12

Pollution Reduction Programs for the 

Chesapeake Bay Watershed

9. Before taking this survey, had you read or heard of programs to 

reduce excess nutrients and sediments in the Chesapeake Bay?

Yes

No

Progress toward each water quality goal will depend on the design of 

the program.

For example,

 Pollution reduction practices close to the Bay will improve water quality in 

the Bay, but not in freshwater lakes farther out in the watershed.

 Planting aquatic grasses will improve aquatic life habitat but will have a 

smaller effect on water clarity and dissolved oxygen than programs focused 

on reducing nutrients and sediment.

Federal and State Agencies are developing programs to meet the goals that have been 

set for water quality and wildlife in the Chesapeake Bay. Any program would be phased 

in over time but would be fully implemented by the year 2025.
13

Paying for Pollution Reduction Programs

Some of the basic things people spend money on would become more expensive. 

For example:

 Higher water bills or increased maintenance costs for septic systems for 

households in the Watershed. For renters, rent or utility bills would increase.

 Higher prices for some agricultural products and other goods for households 

both outside and inside the Watershed, including the area where you live. 

This is because of higher costs for businesses inside the Watershed.

A pollution reduction program, if implemented, would permanently increase the 

cost of living for your household.

The changes required under pollution reduction programs would result in higher costs 

for your household. 
14

Deciding Future Actions

Imagine that you were given the opportunity to vote on future pollution reduction 

programs. 

In the questions that follow, we will ask your opinion about different programs that 

provide different levels of environmental improvements, and cost your household 

different amounts.

You will be asked three questions. In each question you will need to choose the 

program you like best from the following alternatives: 

 The NO FURTHER ACTION option would have no additional pollution reductions 

or costs to your household.

 The two programs labeled PROGRAM A and PROGRAM B differ in how much they 

reduce pollution and how much they cost your household.

An Example Question is on the next page to show you what the questions will 

look like.
15

An Example Question

Conditions in 2025

Current 

Conditions

No Further 

Action Program A Program B

Water Clarity

Average visibility 

of 3 feet

Average visibility of

3 feet

no change

Average visibility of

4 feet

33% increase

Average visibility of

3 feet

67% increase

Blue Crab

250 million crabs

250

million crabs

no change

300

million crabs

20% increase 

275

million crabs

10% increase 

Oysters

3,300 tons

3,300

tons

no change

3,960

tons

20% increase 

4,950

tons

50% increase 

Lake Condition

Average of 3

in <state> 

3

Brown/Green Water

3 feet of visibility

Bass and rough fish 

3

Brown/Green Water

3 feet of visibility

Bass and rough fish

4

Blue/Brown Water

5 feet of visibility

Mostly bass 

Permanent 

Increase in the 

Cost of Living for 

your Household 

Your Vote

Please mark one of 

the boxes to the 

right 

$0 per year 

No Further Action 

$150 per year

or $12.50 per 

month 

Program A 

$200 per year

or $16.67 per 

month

Program B 

Chesapeake Bay and Tidal Waters Lakes

If you do not want A 

or B, mark this box

If you want Program A, 

mark this box

If you want Program B, 

mark this box
16

Consider the Outcomes and the Costs

Please take time to consider both the benefits of the program and the costs to 

your household. Ask yourself if the program outcomes for the Chesapeake Bay 

Watershed are worth the additional cost to your household. 

Paying the costs means you would have less money to spend on other things 

such as food, clothes, going on trips, and even towards resolving other 

environmental problems you care about. 

You would be making a commitment to pay the additional amount every year 

from now on. 

Only you know what is best for you and your household, so please only think of your 

household when deciding whether to vote for NO FURTHER ACTION, PROGRAM A, or 

PROGRAM B.

Please think carefully about how you would actually vote in these situations. We urge 

you to respond as though costs for your household really would go up if the program 

were implemented. Knowing how different people would vote on this program is very 

important for decision makers in state and federal governments.

Weighing Outcomes and Costs
17

Conditions in 2025

Current 

Conditions

No Further 

Action Program A Program B

Water Clarity

Average visibility 

of 3 feet 

Average visibility of

3 feet

no change

Average visibility of

5 feet

83% increase 

Average visibility of

4 feet

50% increase 

Blue Crab

250 million crabs

250 million crabs

no change

280 million crabs

12% increase 

312 million crabs

25% increase 

Oysters

3,300 tons

3,300 tons

no change 

5,250 tons

50% increase 

4,300 tons

23% increase 

Lake Condition

Average of 3

in <state> 

3

Brown/Green Water

3 feet of visibility

Bass and rough fish 

3

Brown/Green Water

3 feet of visibility

Bass and rough fish

4

Blue/Brown Water

5 feet of visibility

Mostly bass 

Permanent 

Increase in the 

Cost of Living for 

your Household 

Your Vote

Please mark one of 

the boxes to the 

right 

$0 per year 

No Further Action 

$180 per year

or $15 per month 

Program A 

$250 per year

or $20.83 per 

month

Program B 

Chesapeake Bay and Tidal Waters Lakes

10. Suppose you could choose from the following three options. Mark

 the box at the bottom to indicate which plan you would vote for.
18

As you vote on the next questions, please remember...

 On the following pages you will see questions that present new sets of program 

options. These options require a different mix of management practices in different 

areas.

 Consider each question separately, imagining that the options are the only ones 

available to choose from.

 The next two questions cannot be compared to each other, or to the question you 

just answered.

 Do not add up effects or costs across different questions. 
19

Chesapeake Bay and Tidal Waters Lakes

11. Now suppose instead that a different set of options were available. 

 Please mark the box at the bottom to indicate which plan you 

 would vote for.

Conditions in 2025

Current 

Conditions

No Further 

Action Program A Program B

Water Clarity

Average visibility 

of 3 feet 

Average visibility of

3 feet

no change

Average visibility of

4 feet

33% increase 

Average visibility of

5 feet

no change 

Blue Crab

250 million crabs

250 million crabs

no change

250 million crabs

no change

280 million crabs

12% increase 

Oysters

3,300 tons

3,300 tons

no change

4,300 tons

30% increase 

5,250 tons

59% increase 

Lake Condition

Average of 3

in <state> 

3

Brown/Green Water

3 feet of visibility

Bass and rough fish 

4

Blue/Brown Water

5 feet of visibility

Mostly Bass 

5

Blue/Brown Water

8 feet of visibility

Bass and walleye

Permanent 

Increase in the 

Cost of Living for 

your Household 

Your Vote

Please mark one of 

the boxes to the 

right 

$0 per year 

No Further Action 

$60 per year

or $5 per month 

Program A 

$40 per year

or $20.83 per 

month

Program B 
20

Chesapeake Bay and Tidal Waters Lakes

12. Now suppose instead that a different set of options were available. 

 Please mark the box at the bottom to indicate which plan you 

 would vote for.

Conditions in 2025

Current 

Conditions

No Further 

Action Program A Program B

Water Clarity

Average visibility 

of 3 feet 

Average visibility of

3 feet

no change

Average visibility of

5 feet

83% increase 

Average visibility of

4 feet

33% increase 

Blue Crab

250 million crabs

250 million crabs

no change

280 million crabs

12% increase 

312 million crabs

25% increase 

Oysters

3,300 tons

3,300 tons

no change

5,250 tons

50% increase 

3,300 tons

no change 

Lake Condition

Average of 3

in <state> 

3

Brown/Green Water

3 feet of visibility

Bass and rough fish 

5

Blue/Brown Water

8 feet of visibility

Bass and walleye 

3

Brown/Green Water

3 feet of visibility

Bass and rough fish 

Permanent 

Increase in the 

Cost of Living for 

your Household 

Your Vote

Please mark one of 

the boxes to the 

right 

$0 per year 

No Further Action 

$60 per year

or $3.33 per month 

Program A 

$180 per year

or $15 per month

Program B 
21

13. For the choices you just made about programs, how much 

 consideration did you give to improvements in lakes compared 

 to the Chesapeake Bay tidal waters? (Please circle one.)

Considered 

Only the 

Lakes

Considered 

Lakes and 

Bay 

Waters 

Equally

Considered 

Only the 

Bay 

Waters

1 2 3 4 5 6 7

14. On a scale from 1 to 5, please rank the following based in order

 of importance when you were choosing between programs. 

Not 

Important

Somewhat 

Important

Very 

Important

Don’t 

Know

Clarity 1 2 3 4 5 DK

Striped Bass 1 2 3 4 5 DK

Blue Crab 1 2 3 4 5 DK

Lake Category 1 2 3 4 5 DK

Cost of the program 1 2 3 4 5 DK
22

15. Aside from costs, which of the following was most important

 in making your decision about which program to vote for? 

 (Please choose one.)

Recreation on the Bay

Recreation in Lakes

Preserving the environment for future generations

Availability and/or price of seafood

Value of property near the Bay

Value of property near Lakes

The possibility of visiting the Bay in the future

The possibility of visiting Lakes in the future

Other __________________
23

16. Thinking about how you voted among programs, please rate

 how much you agree or disagree with each of the following 

 statements. (Circle the number that matches your answer.)

Strongly 

Disagree Neutral Strongly Agree

My choices would have been 

different if the economy in my 

area were better. 1 2 3 4 5 6 7

It is important to improve waters 

in the Chesapeake Bay 

watershed, no matter how high 

the costs.

1 2 3 4 5 6 7

I do not think I should have to 

contribute any amount to the 

restoration of the Chesapeake 

Bay watershed.

1 2 3 4 5 6 7

I am concerned that the 

programs would hurt the 

economy.

1 2 3 4 5 6 7

It was difficult to understand the 

descriptions of the programs. 1 2 3 4 5 6 7

I believe the programs will 

improve the environment as 

described.

1 2 3 4 5 6 7

Some of the programs cost too 

much compared to what they 

would deliver. 1 2 3 4 5 6 7

The changes offered by the 

programs happen too far into the 

future for me to really care 

about.

1 2 3 4 5 6 7

The survey provided me with the 

information I needed to make a 

choice between the options 

shown.

1 2 3 4 5 6 7
24

17. If you voted for NO FURTHER ACTION in any of the choices you

 were given, please let us know how much you agree or disagree

 with each of the following statements. If not, skip to the next 

 question. (Circle the number that matches your answer.)

Strongly 

Disagree Neutral Strongly Agree

Nothing can be done to 

improve the quality of the Bay 

and freshwater lakes.

1 2 3 4 5 6 7

I would not vote for any 

program even if there were no 

costs to my household.

1 2 3 4 5 6 7

I voted for NO FURTHER 

ACTION because the costs were 

too high compared to the 

improvements.

1 2 3 4 5 6 7

I voted for NO FURTHER 

ACTION because I am against 

more government requirements 

and spending.

1 2 3 4 5 6 7
25

Strongly 

Disagree Neutral Strongly Agree

I voted for a program because I 

thought it would also lead to 

better water quality closer to 

my home. 

1 2 3 4 5 6 7

I voted for a program more for 

future generations than for 

myself. 

1 2 3 4 5 6 7

I voted for the program 

because I may visit the 

Chesapeake Bay or lakes in the 

watershed in the next few 

years. 

1 2 3 4 5 6 7

I voted for a program because I 

believe it is our responsibility to 

preserve wildlife and habitat in 

the Bay and Lakes. 

1 2 3 4 5 6 7

I voted for a program so others 

could enjoy the Bay and lakes, 

rather than for myself. 

1 2 3 4 5 6 7

18. If you voted for Program A or Program B in any of the choices

 you were given, please let us know how much you agree or 

 disagree with each of the following statements. If not, skip this 

 question. (Circle the number that matches your answer.)
26

19. Are you a male or female?

Questions about you and your household

Finally, we would like to ask you a few questions about you and your household for 

statistical purposes. The reports prepared for this study will summarize findings 

across all responses and will not associate responses with an individual. Your 

answers will not be saved or stored in a way that can be associated with your name 

or address. You will not be contacted about your responses or this survey. 

Male

Female

20. What is your age?

______ years old

21. What is your current marital status?

Under $25,000

$25,000-$34,999

$35,000-$49,999

$50,000-$74,999

$75,000-$99,999

$100,000-$199,999

$200,000 or more

22. How many children under age 18 are living at your home?

______ children

23. In 2011, what was your total pre-tax household income, including

 all earners in your household?

Single, never married

Married or living with a long-term partner

Separated or divorced

Widowed
27

25. Which of the following best describes the home or apartment you live in?

Owned by you or someone in your household with a mortgage or loan

Owned by you or someone in your household without a mortgage or loan

Rented

Other: _____________________________________________

24. What is the highest degree or level of school you have completed?

No high school diploma

High school diploma or GED

Some college credit or college degree

Some graduate school or professional school credit or a graduate

or professional degree

26. Have you or any member of your family ever worked for any of the

 following industries or jobs? (Please fill in the box next to all that apply.)

Agriculture

Commercial fishing

Tour guide for fishing

No one in my family has worked in these industries

Thank you very much 

for your help.
The public reporting and recordkeeping burden for this collection of information is estimated 

to average 2 hours per response. Send comments on the Agency's need for this 

information, the accuracy of the provided burden estimates, and any suggested methods for 

minimizing respondent burden, including through the use of automated collection techniques 

to the Director, Collection Strategies Division, U.S. Environmental Protection Agency 

(2822T), 1200 Pennsylvania Ave., NW, Washington, D.C. 20460. Include the OMB control 

number in any correspondence. Do not send the materials to this address.
Attachment A. Stated Preference Questionnaire, Attributes are Inputs

& Dual, Constant Baseline
Attachment B. Stated Preference Questionnaire, Attributes are

Endpoints & Dual, Constant Baseline
Attachment C. Stated Preference Questionnaire, Attributes are

Inputs, Declining Baseline
Chesapeake Bay Stated Preference

The Future of the Chesapeake Bay: 

What’s Your Opinion?
About this survey

We are conducting this survey about the Chesapeake Bay Watershed (that is 

both the Bay and the lands around it). The survey’s purpose is to collect 

information to help policymakers make decisions that affect water quality, wildlife 

and recreation. 

In this survey, the first 

questions are about: 

 Your experiences with the 

Chesapeake Bay Watershed

 Your thoughts about the 

Watershed’s current 

conditions

Then the survey describes: 

 Different plans that could 

affect the Watershed

 What changes might result 

from each plan

 How a plan might affect 

households like yours 

We want to know which of these plans you would vote for. Finally we ask a few 

questions about your opinions on the survey’s topics, and some background 

information about your household.

Your responses to this survey will help to inform policymakers how the public feels 

about plans that would affect the Chesapeake Bay Watershed and your household. 

This is your opportunity to voice your opinions about these plans. Please 

take the time to read the information in this survey and answer the questions 

carefully.

All information and responses will be kept confidential. Background information in 

this survey was provided by the National Marine Fisheries Service, U.S. 

Environmental Protection Agency, and other state and federal offices.

1
The Chesapeake Bay Watershed

The Chesapeake Bay Watershed is shown below in green. 

The Watershed spreads over parts of six states and Washington DC. It contains 150 

major rivers and thousands of lakes, small rivers and streams. Water draining 

from these lands enters these waterways and eventually the Chesapeake Bay. 

The Chesapeake Bay Watershed includes two types of water bodies, with different 

characteristics and potential problems.

2

Tidal waters include the Bay 

itself and parts of some rivers and streams 

that flow into it. Tidal waters are salty and 

support different types of wildlife than 

freshwater, but are also used to swim, 

boat, and fish.

Freshwater lakes and 

rivers support wildlife and 

are used for swimming, 

boating, and fishing, among 

other activities.
3

Lakes and rivers 

in the Watershed

Chesapeake Bay 

tidal waters

Swimming, water skiing, or 

jet skiing

□ □

Fishing □ □

Boating, including sailing, 

motor boating and kayaking

□ □

Viewing nature, including 

bird watching □ □

Biking or walking on trails 

along the water □ □

Other 

_______________________

□ □

1. Over the last five years, which of these activities have you done 

in any freshwater lakes or rivers, or in the Bay’s tidal waters?
2. In the last five years, about how often have you visited 

watershed lakes or rivers, or the Chesapeake Bay Tidal waters 

to do any of these activities? 

In each column please check the statement that best describes 

your experience.

4

Lakes and rivers 

in the Watershed

Chesapeake Bay 

tidal waters

I have never visited □ □

I have not visited in the last 

5 years □ □

Once over the last 5 years □ □

A few times over the last 5 

years 

□ □

About once a year □ □

2 to 10 times a year □ □

More than 10 times a year □ □
5

Water Quality in the 

Chesapeake Bay Watershed

For the following questions, check the box next to your answer.

3. Before taking this survey, were you aware that too much 

sediment or nutrients can harm wildlife habitats?

Yes

No

4. Have you previously heard of any sources of excess nutrients and 

sediments in the Chesapeake Bay Watershed? 

Yes

No

Pollutants in the Chesapeake Bay Watershed degrade the quality of the 

water, and can affect wildlife habitat and recreational opportunities. Two key 

pollutants are nutrients and sediments.

 Nutrients, such as nitrogen and phosphorous occur naturally in the 

watershed, and are essential for healthy ecosystems. But excess nutrients 

(from fertilizers, livestock manure, and wastewater treatment plants) can lead 

to algae that deprives fish and plants of oxygen.

 Sediment. Some loose soil naturally settles to the bottom of water bodies. 

Too much sediment makes the water murky and harms aquatic plants and 

animals. Paved surfaces and some farming practices increase soil erosion, 

causing too much sediment to enter rivers and streams that flow into the Bay.
6

Reducing excess nutrients and sediment in 

the Chesapeake Bay Watershed

Changing the way land, storm water, and waste water are managed can improve 

water quality in the Chesapeake Bay Watershed. As a result, wildlife habitat and 

recreation opportunities will improve in lakes, rivers and the Bay.

Practices to Improve Water Quality

 Changing the way farmers dispose of livestock manure and till their land 

 Installing better equipment at wastewater treatment facilities

 Restoring streams and wetlands and paving fewer surfaces to slow storm 

water runoff 

 Restoring oyster reefs and bay grass beds to filter water and improve 

fish habitat

Water Quality Improvements

 Clearer water because less nutrients and sediment enter the water 

 More aquatic grasses will grow because clearer water allows more sunlight to 

reach them

 Smaller, less frequent algae blooms 

 More oxygen in the water because of reduced algae 

Ecosystem and Recreation Improvements

 Cleaner and healthier aquatic environment 

 More game fish and crabs for recreational fishermen

 More enjoyable boating and swimming 

 More oysters and bay grass beds will provide habitat for a variety of fish and 

shellfish while also improving water quality
5. How do you think environmental conditions have changed 

in lakes and rivers closest to your home and in the 

Chesapeake Bay? (Please check one box in each column.)

7

Lakes and Rivers Chesapeake Bay

Better than they were 10 years 

ago 

□ □

About the same as they were 

10 years ago □ □

Worse than they were 10 years 

ago □ □

Don’t know □ □

Strongly

Disagree

Neutral Strongly 

Agree

Don’t 

Know

I am concerned about current 

pollution levels in lakes and rivers. 1 2 3 4 5 DK

I am concerned about current 

pollution levels in the Chesapeake 

Bay.

1 2 3 4 5 DK

Some decrease in environmental 

quality is inevitable as our economy 

grows.

1 2 3 4 5 DK

6. On a scale from 1 to 5, please circle the number that indicates how 

much you agree or disagree with each statement. 
8

Environmental Quality in the 

Chesapeake Bay

For the Chesapeake Bay, environmental goals are based on several 

measurements of environmental quality and aquatic life:

Dissolved Oxygen

 Dissolved oxygen refers to oxygen in the water, something all aquatic life 

needs

 Dissolved oxygen goals were set for different areas of the Chesapeake Bay 

based on the types of aquatic life that live there

Water Clarity

 Water clarity is measured by how far one can see into the water

 Clearer water means more sunlight for underwater plants to grow

 People who go swimming or boating may prefer clearer water

Amount of Aquatic Grass Habitat

 Aquatic grasses provide food and habitat for other aquatic life 

Different areas of the Chesapeake Bay provide different types of wildlife habitat and 

recreation opportunities. 

State governments and federal agencies have been working with scientists to develop 

environmental goals for water quality in the Chesapeake Bay.

7. Before taking this survey, had you read or heard about any of 

these environmental goals for dissolved oxygen, water clarity, or 

aquatic grasses? 

Yes

No
9

Environmental Quality in the 

Chesapeake Bay

State and local governments already have some pollution reduction programs in place 

but some additional actions are being considered. Scientists studying the Chesapeake 

Bay Watershed monitor its health and predict changes in water quality under different 

pollution reduction programs. 

The table here shows both current conditions and predicted conditions in 2025

for the Chesapeake Bay tidal waters if no further action is taken to reduce nutrients 

and sediment.

Current Conditions Conditions in 2025 with 

no further action *

Dissolved Oxygen (DO)

Percent of waters meeting 

DO goals

Some plans being 

considered would achieve 

100% in 2025

40 %

of the Bay areas

meet DO goals 

33 %

of the Bay areas

will meet DO goals

Water Clarity 

How many feet you can see 

down into the water.

Average visibility of

3 feet

Average visibility of

1.5 feet

50% decrease 

Aquatic Grasses 

Acres of underwater grasses 

on the Bay floor.

Scientists recommend 

185,000 acres be restored 

based on historic levels 

80,000 acres 56,000 acres

30% decrease 

*All forecasts for the year 2025 are based on monitoring data from the Chesapeake Bay Watershed 

and Estuary Models developed by the Chesapeake Bay Program Office of the EPA in conjunction with 

state and federal partners.
10

Environmental Quality in the Lakes

Scientists studying the effect of nutrient pollution on lakes categorize them by how 

much algae grows in the water.

High amounts of algae:

 Changes the color of the water and reduces visibility. 

 Results in fewer game fish like bass and trout and more rough fish like carp and 

catfish.

Lake 

Condition

Water 

Color

Water 

Clarity 

Most Common Types 

of Fish

1 Green Less than 1 foot Mostly rough fish

2 Green About 1 foot Mostly rough fish

3 Brown/Green About 3 feet Bass and rough fish

4 Blue/Brown About 5 feet Mostly bass

5 Blue/Brown About 8 feet Mostly bass and some 

walleye

6 Blue About 15 feet Bass, walleye, and trout

7 Blue More than 20 feet Bass, walleye, and trout

8. What condition do you think best describes the lakes closest to your 

home? (Please circle one.) 

1 2 3 4 5 6 7

Don’t 

Know
11

Lakes in the Chesapeake Bay Watershed

Some important things to note: 

 Meeting the goals for water quality in the Chesapeake Bay would also affect 

freshwater bodies of the watershed. 

 Nutrients have a small effect on many rivers because the water is constantly 

moving. 

 Reducing the amount of nutrients entering lakes will improve the appearance of 

the water and change the ecological conditions. 

The table below shows the current condition and conditions in 2025 that scientists 

predict for lakes in the part of the watershed in your state if no further actions are 

taken to reduce nutrient and sediment pollution.

The table shows the average condition for <state> lakes within the Watershed. This is 

an average, so individual lakes near you could be higher or lower. 

Current Conditions Conditions in 2025 with

no additional actions*

Average Condition 

of <State> Lakes in 

the Watershed

3

Brown/Green Water

3 feet of visibility

Bass and rough fish 

2

Green Water

1 foot of visibility

Mostly rough fish 

*All forecasts for the year 2025 are based on measures from the Northeast Lakes Model developed by 

the Office of Research and Development of the EPA.
12

Pollution Reduction Programs for the 

Chesapeake Bay Watershed

9. Before taking this survey, had you read or heard of programs to 

reduce excess nutrients and sediments in the Chesapeake Bay?

Yes

No

Progress toward each water quality goal will depend on the design of 

the program.

For example,

 Pollution reduction practices close to the Bay will improve water quality in 

the Bay, but not in freshwater lakes farther out in the watershed.

 Planting aquatic grasses will improve aquatic life habitat but will have a 

smaller effect on water clarity and dissolved oxygen than programs focused 

on reducing nutrients and sediment.

Federal and State Agencies are developing programs to meet the goals that have been 

set for water quality and wildlife in the Chesapeake Bay. Any program would be phased 

in over time but would be fully implemented by the year 2025.
13

Paying for Pollution Reduction Programs

Some of the basic things people spend money on would become more expensive. 

For example:

 Higher water bills or increased maintenance costs for septic systems for 

households in the Watershed. For renters, rent or utility bills would increase.

 Higher prices for some agricultural products and other goods for households 

both outside and inside the Watershed, including the area where you live. 

This is because of higher costs for businesses inside the Watershed.

A pollution reduction program, if implemented, would permanently increase the 

cost of living for your household.

The changes required under pollution reduction programs would result in higher costs 

for your household. 
14

Deciding Future Actions

Imagine that you were given the opportunity to vote on future pollution reduction 

programs. 

In the questions that follow, we will ask your opinion about different programs that 

provide different levels of environmental improvements, and cost your household 

different amounts.

You will be asked three questions. In each question you will need to choose the 

program you like best from the following alternatives: 

 The NO FURTHER ACTION option would have no additional pollution reductions 

or costs to your household.

 The two programs labeled PROGRAM A and PROGRAM B differ in how much they 

reduce pollution and how much they cost your household.

An Example Question is on the next page to show you what the questions will 

look like.
15

An Example Question

Conditions in 2025

Current 

Conditions

No Further 

Action Program A Program B

Dissolved 

Oxygen

40% of the Bay

areas will meet DO 

goals 

33%

of the Bay areas

will meet DO goals 

50%

of the Bay areas

will meet DO goals

70%

of the Bay areas

will meet DO goals

Water Clarity

Average visibility 

of 3 feet 

Average visibility of

1.5 feet

50% decrease 

Average visibility of

4 feet

33% increase 

Average visibility of

3 feet

67% increase 

Aquatic 

Grasses

Coverage is 

80,000 acres

56,000 acres

30% decrease 

100,000 acres

25% increase 

150,000 acres

50% increase 

Lake Condition

Average of 3

in <state> 

2

Green Water

1 foot of visibility

Mostly rough fish 

4

Blue/Brown Water

5 feet of visibility

Mostly Bass

3

Brown/Green Water

3 feet of visibility

Bass and rough fish 

Permanent 

Increase in the 

Cost of Living for 

your Household 

Your Vote

Please mark one of 

the boxes to the 

right 

$0 per year 

No Further Action 

$36 per year

or $3 per month 

Program A 

$50 per year

or $4.16 per month

Program B 

Chesapeake Bay and Tidal Waters Lakes

If you do not want A 

or B, mark this box

If you want Program A, 

mark this box

If you want Program B, 

mark this box
16

Consider the Outcomes and the Costs

Please take time to consider both the benefits of the program and the costs to 

your household. Ask yourself if the program outcomes for the Chesapeake Bay 

Watershed are worth the additional cost to your household. 

Paying the costs means you would have less money to spend on other things 

such as food, clothes, going on trips, and even towards resolving other 

environmental problems you care about. 

You would be making a commitment to pay the additional amount every year 

from now on. 

Only you know what is best for you and your household, so please only think of your 

household when deciding whether to vote for NO FURTHER ACTION, PROGRAM A, or 

PROGRAM B.

Please think carefully about how you would actually vote in these situations. We urge 

you to respond as though costs for your household really would go up if the program 

were implemented. Knowing how different people would vote on this program is very 

important for decision makers in state and federal governments.

Weighing Outcomes and Costs
17

Conditions in 2025

Current 

Conditions

No Further 

Action Program A Program B

Dissolved 

Oxygen

40% of the Bay

areas will meet DO 

goals 

33%

of the Bay areas

will meet DO goals 

85%

of the Bay areas

will meet DO goals

100%

of the Bay areas

will meet DO goals

Water Clarity

Average visibility 

of 3 feet 

Average visibility of

1.5 feet

50% decrease 

Average visibility of

5 feet

83% increase 

Average visibility of

4 feet

50% increase 

Aquatic 

Grasses

Coverage is 

80,000 acres

56,000 acres

30% decrease 

130,000 acres

25% increase 

165,000 acres

50% increase 

Lake Condition

Average of 3

in <state> 

2

Green Water

1 foot of visibility

Mostly rough fish 

5

Blue/Brown Water

8 feet of visibility

Bass and walleye

4

Blue/Brown Water

5 feet of visibility

Mostly bass 

Permanent 

Increase in the 

Cost of Living for 

your Household 

Your Vote

Please mark one of 

the boxes to the 

right 

$0 per year 

No Further Action 

$250 per year

or $20.83 per 

month 

Program A 

$180 per year

or $15 per month

Program B 

Chesapeake Bay and Tidal Waters Lakes

10. Suppose you could choose from the following three options. Mark

 the box at the bottom to indicate which plan you would vote for.
18

As you vote on the next questions please remember...

 On the following pages you will see questions that present new sets of program 

options. These options require a different mix of management practices in different 

areas.

 Consider each question separately, imagining that the options are the only ones 

available to choose from.

 The next two questions cannot be compared to each other, or to the question you 

just answered.

 Do not add up effects or costs across different questions. 
19

Conditions in 2025

Current 

Conditions

No Further 

Action Program A Program B

Dissolved 

Oxygen

40% of the Bay

areas will meet DO 

goals 

33%

of the Bay areas

will meet DO goals 

85%

of the Bay areas

will meet DO goals

40%

of the Bay areas

will meet DO goals

Water Clarity

Average visibility 

of 3 feet 

Average visibility of

1.5 feet

50% decrease 

Average visibility of

4 feet

33% increase 

Average visibility of

5 feet

no change 

Aquatic 

Grasses

Coverage is 

80,000 acres

56,000 acres

30% decrease 

130,000 acres

25% increase 

80,000 acres

50% increase 

Lake Condition

Average of 3

in <state>

2

Green Water

1 foot of visibility

Mostly rough fish 

3

Brown/Green Water

3 feet of visibility

Bass and rough fish

4

Blue/Brown Water

5 feet of visibility

Mostly bass 

Permanent 

Increase in the 

Cost of Living for 

your Household 

Your Vote

Please mark one of 

the boxes to the 

right 

$0 per year 

No Further Action 

$40 per year

or $3.33 per month 

Program A 

$60 per year

or $5 per month

Program B 

Chesapeake Bay and Tidal Waters Lakes

11. Now suppose instead that a different set of options were available. 

 Please mark the box at the bottom to indicate which plan you 

 would vote for.
20

Conditions in 2025

Current 

Conditions

No Further 

Action Program A Program B

Dissolved 

Oxygen

40% of the Bay

areas will meet DO 

goals 

33%

of the Bay areas

will meet DO goals 

100%

of the Bay areas

will meet DO goals

85%

of the Bay areas

will meet DO goals

Water Clarity

Average visibility 

of 3 feet 

Average visibility of

1.5 feet

50% decrease 

Average visibility of

5 feet

83% increase 

Average visibility of

4 feet

33% increase 

Aquatic 

Grasses

Coverage is 

80,000 acres

56,000 acres

30% decrease 

130,000 acres

20% increase 

165,000 acres

75% increase 

Lake Condition

Average of 3

in <state>

2

Green Water

1 foot of visibility

Mostly rough fish 

5

Blue/Brown Water

8 feet of visibility

Bass and walleye

4

Blue/Brown Water

5 feet of visibility

Mostly bass 

Permanent 

Increase in the 

Cost of Living for 

your Household 

Your Vote

Please mark one of 

the boxes to the 

right 

$0 per year 

No Further Action 

$370 per year

or $30.83 per 

month 

Program A 

$180 per year

or $15 per month

Program B 

Chesapeake Bay and Tidal Waters Lakes

12. Now suppose instead that a different set of options were available. 

 Please mark the box at the bottom to indicate which plan you 

 would vote for.
21

13. For the choices you just made about programs, how much 

 consideration did you give to improvements in lakes compared 

 to the Chesapeake Bay tidal waters? (Please circle one.)

Considered 

Only the 

Lakes

Considered 

Lakes and 

Bay 

Waters 

Equally

Considered 

Only the 

Bay 

Waters

1 2 3 4 5 6 7

14. On a scale from 1 to 5, please rank the following based in order

 of importance when you were choosing between programs. 

Not 

Important

Somewhat 

Important

Very 

Important

Don’t 

Know

Dissolved oxygen 1 2 3 4 5 DK

Water clarity 1 2 3 4 5 DK

Aquatic grass coverage 1 2 3 4 5 DK

Lake category 1 2 3 4 5 DK

Cost of the program 1 2 3 4 5 DK
22

15. Aside from costs, which of the following was most important

 in making your decision about which program to vote for? 

 (Please choose one.)

Recreation on the Bay

Recreation in Lakes

Preserving the environment for future generations

Availability and/or price of seafood

Value of property near the Bay

Value of property near Lakes

The possibility of visiting the Bay in the future

The possibility of visiting Lakes in the future

Other __________________
23

16. Thinking about how you voted among programs, please rate

 how much you agree or disagree with each of the following 

 statements. (Circle the number that matches your answer.)

Strongly 

Disagree Neutral Strongly Agree

My choices would have been 

different if the economy in my 

area were better. 1 2 3 4 5 6 7

It is important to improve waters 

in the Chesapeake Bay 

watershed, no matter how high 

the costs.

1 2 3 4 5 6 7

I do not think I should have to 

contribute any amount to the 

restoration of the Chesapeake 

Bay watershed.

1 2 3 4 5 6 7

I am concerned that the 

programs would hurt the 

economy.

1 2 3 4 5 6 7

It was difficult to understand the 

descriptions of the programs. 1 2 3 4 5 6 7

I believe the programs will 

improve the environment as 

described.

1 2 3 4 5 6 7

Some of the programs cost too 

much compared to what they 

would deliver. 1 2 3 4 5 6 7

The changes offered by the 

programs happen too far into the 

future for me to really care 

about.

1 2 3 4 5 6 7

The survey provided me with the 

information I needed to make a 

choice between the options 

shown.

1 2 3 4 5 6 7
24

17. If you voted for NO FURTHER ACTION in any of the choices you

 were given, please let us know how much you agree or disagree

 with each of the following statements. If not, skip to the next 

 question. (Circle the number that matches your answer.)

Strongly 

Disagree Neutral Strongly Agree

Nothing can be done to 

improve the quality of the Bay 

and freshwater lakes.

1 2 3 4 5 6 7

I would not vote for any 

program even if there were no 

costs to my household.

1 2 3 4 5 6 7

I voted for NO FURTHER 

ACTION because the costs were 

too high compared to the 

improvements.

1 2 3 4 5 6 7

I voted for NO FURTHER 

ACTION because I am against 

more government requirements 

and spending.

1 2 3 4 5 6 7
25

Strongly 

Disagree Neutral Strongly Agree

I voted for a program because I 

thought it would also lead to 

better water quality closer to 

my home. 

1 2 3 4 5 6 7

I voted for a program more for 

future generations than for 

myself. 

1 2 3 4 5 6 7

I voted for the program 

because I may visit the 

Chesapeake Bay or lakes in the 

watershed in the next few 

years. 

1 2 3 4 5 6 7

I voted for a program because I 

believe it is our responsibility to 

preserve wildlife and habitat in 

the Bay and Lakes. 

1 2 3 4 5 6 7

I voted for a program so others 

could enjoy the Bay and lakes, 

rather than for myself. 

1 2 3 4 5 6 7

18. If you voted for Program A or Program B in any of the choices

 you were given, please let us know how much you agree or 

 disagree with each of the following statements. If not, skip this 

 question. (Circle the number that matches your answer.)
26

19. Are you male or female?

Questions about you and your household

Finally, we would like to ask you a few questions about you and your household for 

statistical purposes. The reports prepared for this study will summarize findings 

across all responses and will not associate responses with an individual. Your 

answers will not be saved or stored in a way that can be associated with your name 

or address. You will not be contacted about your responses or this survey. 

Male

Female

20. What is your age?

______ years old

21. What is your current marital status?

Under $25,000

$25,000-$34,999

$35,000-$49,999

$50,000-$74,999

$75,000-$99,999

$100,000-$199,999

$200,000 or more

22. How many children under age 18 are living at your home?

______ children

23. In 2011, what was your total pre-tax household income, including

 all earners in your household?

Single, never married

Married or living with a long-term partner

Separated or divorced

Widowed
27

25. Which of the following best describes the home or apartment you live in?

Owned by you or someone in your household with a mortgage or loan

Owned by you or someone in your household without a mortgage or loan

Rented

Other: _____________________________________________

24. What is the highest degree or level of school you have completed?

No high school diploma

High school diploma or GED

Some college credit or college degree

Some graduate school or professional school credit or a graduate

or professional degree

26. Have you or any member of your family ever worked for any of the

 following industries or jobs? (Please fill in the box next to all that apply.)

Agriculture

Commercial fishing

Tour guide for fishing

No one in my family has worked in these industries

Thank you very much 

for your help.
The public reporting and recordkeeping burden for this collection of information is estimated 

to average 2 hours per response. Send comments on the Agency's need for this 

information, the accuracy of the provided burden estimates, and any suggested methods for 

minimizing respondent burden, including through the use of automated collection techniques 

to the Director, Collection Strategies Division, U.S. Environmental Protection Agency 

(2822T), 1200 Pennsylvania Ave., NW, Washington, D.C. 20460. Include the OMB control 

number in any correspondence. Do not send the materials to this address.
Attachment D. Stated Preference Questionnaire, Attributes are

Endpoints & Dual, Declining Baseline
Chesapeake Bay Stated Preference

The Future of the Chesapeake Bay: 

What’s Your Opinion?
About this survey

We are conducting this survey about the Chesapeake Bay Watershed (that is 

both the Bay and the lands around it). The survey’s purpose is to collect 

information to help policymakers make decisions that affect water quality, wildlife 

and recreation. 

In this survey, the first 

questions are about: 

 Your experiences with the 

Chesapeake Bay Watershed

 Your thoughts about the 

Watershed’s current 

conditions

Then the survey describes: 

 Different plans that could 

affect the Watershed

 What changes might result 

from each plan

 How a plan might affect 

households like yours 

We want to know which of these plans you would vote for. Finally we ask a few 

questions about your opinions on the survey’s topics, and some background 

information about your household.

Your responses to this survey will help to inform policymakers how the public feels 

about plans that would affect the Chesapeake Bay Watershed and your household. 

This is your opportunity to voice your opinions about these plans. Please 

take the time to read the information in this survey and answer the questions 

carefully.

All information and responses will be kept confidential. Background information in 

this survey was provided by the National Marine Fisheries Service, U.S. 

Environmental Protection Agency, and other state and federal offices.

1
The Chesapeake Bay Watershed

The Chesapeake Bay Watershed is shown below in green. 

The Watershed spreads over parts of six states and Washington DC. It contains 150 

major rivers and thousands of lakes, small rivers and streams. Water draining 

from these lands enters these waterways and eventually the Chesapeake Bay. 

The Chesapeake Bay Watershed includes two types of water bodies, with different 

characteristics and potential problems.

2

Tidal waters include the Bay 

itself and parts of some rivers and streams 

that flow into it. Tidal waters are salty and 

support different types of wildlife than 

freshwater, but are also used to swim, 

boat, and fish.

Freshwater lakes and 

rivers support wildlife and 

are used for swimming, 

boating, and fishing, among 

other activities.
3

Lakes and rivers 

in the Watershed

Chesapeake Bay 

tidal waters

Swimming, water skiing, or 

jet skiing

□ □

Fishing □ □

Boating, including sailing, 

motor boating and kayaking

□ □

Viewing nature, including 

bird watching □ □

Biking or walking on trails 

along the water □ □

Other 

_______________________

□ □

1. Over the last five years, which of these activities have you done 

in any freshwater lakes or rivers, or in the Bay’s tidal waters?
2. In the last five years, about how often have you visited 

watershed lakes or rivers, or the Chesapeake Bay Tidal waters 

to do any of these activities? 

In each column please check the statement that best describes 

your experience.

4

Lakes and rivers 

in the Watershed

Chesapeake Bay 

tidal waters

I have never visited □ □

I have not visited in the last 

5 years □ □

Once over the last 5 years □ □

A few times over the last 5 

years 

□ □

About once a year □ □

2 to 10 times a year □ □

More than 10 times a year □ □
5

Water Quality in the 

Chesapeake Bay Watershed

For the following questions, check the box next to your answer.

3. Before taking this survey, were you aware that too much 

sediment or nutrients can harm wildlife habitats?

Yes

No

4. Have you previously heard of any sources of excess nutrients and 

sediments in the Chesapeake Bay Watershed? 

Yes

No

Pollutants in the Chesapeake Bay Watershed degrade the quality of the 

water, and can affect wildlife habitat and recreational opportunities. Two key 

pollutants are nutrients and sediments.

 Nutrients, such as nitrogen and phosphorous occur naturally in the 

watershed, and are essential for healthy ecosystems. But excess nutrients 

(from fertilizers, livestock manure, and wastewater treatment plants) can lead 

to algae that deprives fish and plants of oxygen.

 Sediment. Some loose soil naturally settles to the bottom of water bodies. 

Too much sediment makes the water murky and harms aquatic plants and 

animals. Paved surfaces and some farming practices increase soil erosion, 

causing too much sediment to enter rivers and streams that flow into the Bay.
6

Reducing excess nutrients and sediment in 

the Chesapeake Bay Watershed

Changing the way land, storm water, and waste water are managed can improve 

water quality in the Chesapeake Bay Watershed. As a result, wildlife habitat and 

recreation opportunities will improve in lakes, rivers and the Bay.

Practices to Improve Water Quality

 Changing the way farmers dispose of livestock manure and till their land 

 Installing better equipment at wastewater treatment facilities

 Restoring streams and wetlands and paving fewer surfaces to slow storm 

water runoff 

 Restoring oyster reefs and bay grass beds to filter water and improve 

fish habitat

Water Quality Improvements

 Clearer water because less nutrients and sediment enter the water 

 More aquatic grasses will grow because clearer water allows more sunlight to 

reach them

 Smaller, less frequent algae blooms 

 More oxygen in the water because of reduced algae 

Ecosystem and Recreation Improvements

 Cleaner and healthier aquatic environment 

 More game fish and crabs for recreational fishermen

 More enjoyable boating and swimming 

 More oysters and bay grass beds will provide habitat for a variety of fish and 

shellfish while also improving water quality
5. How do you think environmental conditions have changed 

in lakes and rivers closest to your home and in the 

Chesapeake Bay? (Please check one box in each column.)

7

Lakes and rivers Chesapeake Bay

Better than they were 10 years 

ago 

□ □

About the same as they were 

10 years ago □ □

Worse than they were 10 years 

ago □ □

Don’t know □ □

Strongly

Disagree

Neutral Strongly 

Agree

Don’t 

Know

I am concerned about current 

pollution levels in lakes and rivers. 1 2 3 4 5 DK

I am concerned about current 

pollution levels in the Chesapeake 

Bay.

1 2 3 4 5 DK

Some decrease in environmental 

quality is inevitable as our economy 

grows.

1 2 3 4 5 DK

6. On a scale from 1 to 5, please circle the number that indicates how 

much you agree or disagree with each statement. 
8

Environmental Quality in the 

Chesapeake Bay

For the Chesapeake Bay, environmental goals are based on several 

measurements of water quality and aquatic life:

Water Clarity

 Measured by how far one can see into the water

 People who go swimming or boating my prefer clearer water

 Clearer water means more sunlight for underwater plants to grow

Blue Crab

 A key ecological species and popular shellfish for recreational fishermen

Oysters

 Their shells form reefs that provide habitat for other aquatic life

 As “filter feeders” they clean the waters of the Bay 

Different areas of the Chesapeake Bay provide different types of wildlife habitat and 

recreation opportunities. 

State governments and federal agencies have been working with scientists to develop 

environmental goals for improving the quality of the Chesapeake Bay.

7. Before taking this survey, had you read or heard about any of 

these environmental goals for water clarity in the Chesapeake Bay

or for any Chesapeake Bay species? 

Yes

No
9

Environmental Quality in the 

Chesapeake Bay

State and local governments already have some pollution reduction programs in place 

but some additional actions are being considered. Scientists studying the Chesapeake 

Bay Watershed monitor its health and predict changes in water quality under different 

pollution reduction programs. 

The table here shows both current conditions and predicted conditions in 2025

for the Chesapeake Bay tidal waters if no further action is taken to reduce nutrients 

and sediment.

Current Conditions Conditions in 2025 with 

no further action *

Water Clarity

How many feet you can see 

down into the water

Average visibility of

3 feet

Average visibility of

1.5 feet

50% decrease 

Blue Crabs 

Goal is 415 million crabs

250

million crabs

235

million crabs

6% decrease 

Oysters 

Goal is 35,000 tons

3,300

tons

2,700

tons

18% decrease 

*All forecasts for the year 2025 are based on monitoring data from the Chesapeake Bay Watershed 

and Estuary Models developed by the Chesapeake Bay Program Office of the EPA in conjunction with 

state and federal partners.
10

Environmental Quality in the Lakes

Scientists studying the effect of nutrient pollution on lakes categorize them by how 

much algae grows in the water.

High amounts of algae:

 Changes the color of the water and reduces visibility. 

 Results in fewer game fish like bass and trout and more rough fish like carp and 

catfish.

Lake 

Condition

Water 

Color

Water 

Clarity 

Most Common Types 

of Fish

1 Green Less than 1 foot Mostly rough fish

2 Green About 1 foot Mostly rough fish

3 Brown/Green About 3 feet Bass and rough fish

4 Blue/Brown About 5 feet Mostly bass

5 Blue/Brown About 8 feet Mostly bass and some 

walleye

6 Blue About 15 feet Bass, walleye, and trout

7 Blue More than 20 feet Bass, walleye, and trout

8. What condition do you think best describes the lakes closest to your 

home? (Please circle one.) 

1 2 3 4 5 6 7

Don’t 

Know
11

Lakes in the Chesapeake Bay Watershed

Some important things to note: 

 Meeting the goals for water quality in the Chesapeake Bay would also affect 

freshwater bodies of the watershed. 

 Nutrients have a small effect on many rivers because the water is constantly 

moving. 

 Reducing the amount of nutrients entering lakes will improve the appearance of 

the water and change the ecological conditions. 

The table below shows current conditions and conditions in 2025 that scientists 

predict for lakes in the part of the watershed in your state if no further actions are 

taken to reduce nutrient and sediment pollution.

The table shows the average condition for <state> lakes within the Watershed. This is 

an average, so individual lakes near you could be higher or lower. 

Current Conditions Conditions in 2025 with

no additional actions*

Average Condition 

of <State> Lakes in 

the Watershed

3

Brown/Green Water

3 feet of visibility

Bass and rough fish 

2

Green Water

1 foot of visibility

Mostly rough fish 

*All forecasts for the year 2025 are based on measures from the Northeast Lakes Model developed by 

the Office of Research and Development of the EPA.
12

Pollution Reduction Programs for the 

Chesapeake Bay Watershed

9. Before taking this survey, had you read or heard of programs to 

reduce excess nutrients and sediments in the Chesapeake Bay?

Yes

No

Progress toward each water quality goal will depend on the design of 

the program.

For example,

 Pollution reduction practices close to the Bay will improve water quality in 

the Bay, but not in freshwater lakes farther out in the watershed.

 Planting aquatic grasses will improve aquatic life habitat but will have a 

smaller effect on water clarity and dissolved oxygen than programs focused 

on reducing nutrients and sediment.

Federal and State Agencies are developing programs to meet the goals that have been 

set for water quality and wildlife in the Chesapeake Bay. Any program would be phased 

in over time but would be fully implemented by the year 2025.
13

Paying for Pollution Reduction Programs

Some of the basic things people spend money on would become more expensive. 

For example:

 Higher water bills or increased maintenance costs for septic systems for 

households in the Watershed. For renters, rent or utility bills would increase.

 Higher prices for some agricultural products and other goods for households 

both outside and inside the Watershed, including the area where you live. 

This is because of higher costs for businesses inside the Watershed.

A pollution reduction program, if implemented, would permanently increase the 

cost of living for your household.

The changes required under pollution reduction programs would result in higher costs 

for your household. 
14

Deciding Future Actions

Imagine that you were given the opportunity to vote on future pollution reduction 

programs. 

In the questions that follow, we will ask your opinion about different programs that 

provide different levels of environmental improvements, and cost your household 

different amounts.

You will be asked three questions. In each question you will need to choose the 

program you like best from the following alternatives: 

 The NO FURTHER ACTION option would have no additional pollution reductions 

or costs to your household.

 The two programs labeled PROGRAM A and PROGRAM B differ in how much they 

reduce pollution and how much they cost your household.

An Example Question is on the next page to show you what the questions will 

look like.
15

An Example Question

Conditions in 2025

Current 

Conditions

No Further 

Action Program A Program B

Water Clarity

Average visibility 

of 3 feet

Average visibility of

1.5 feet

50% decrease

Average visibility of

4 feet

33% increase

Average visibility of

3 feet

67% increase

Blue Crab

250 million crabs

235

million crabs

6% decrease 

300

million crabs

20% increase 

275

million crabs

10% increase 

Oysters

3,300 tons

2,700

tons

18% decrease 

3,960

tons

20% increase 

4,950

tons

50% increase 

Lake Condition

Average of 3

in <state> 

2

Green Water

1 foot of visibility

Mostly rough fish 

3

Brown/Green Water

3 feet of visibility

Bass and rough fish

4

Blue/Brown Water

5 feet of visibility

Mostly bass 

Permanent 

Increase in the 

Cost of Living for 

your Household 

Your Vote

Please mark one of 

the boxes to the 

right 

$0 per year 

No Further Action 

$150 per year

or $12.50 per 

month 

Program A 

$200 per year

or $16.67 per 

month

Program B 

Chesapeake Bay and Tidal Waters Lakes

If you do not want A 

or B, mark this box

If you want Program A, 

mark this box

If you want Program B, 

mark this box
16

Consider the Outcomes and the Costs

Please take time to consider both the benefits of the program and the costs to 

your household. Ask yourself if the program outcomes for the Chesapeake Bay 

Watershed are worth the additional cost to your household. 

Paying the costs means you would have less money to spend on other things 

such as food, clothes, going on trips, and even towards resolving other 

environmental problems you care about. 

You would be making a commitment to pay the additional amount every year 

from now on. 

Only you know what is best for you and your household, so please only think of your 

household when deciding whether to vote for NO FURTHER ACTION, PROGRAM A, or 

PROGRAM B.

Please think carefully about how you would actually vote in these situations. We urge 

you to respond as though costs for your household really would go up if the program 

were implemented. Knowing how different people would vote on this program is very 

important for decision makers in state and federal governments.

Weighing Outcomes and Costs
17

Conditions in 2025

Current 

Conditions

No Further 

Action Program A Program B

Water Clarity

Average visibility 

of 3 feet 

Average visibility of

1.5 feet

50% decrease 

Average visibility of

5 feet

83% increase 

Average visibility of

4 feet

50% increase 

Blue Crab

250 million crabs

235 million crabs

6% decrease 

280 million crabs

12% increase 

312 million crabs

25% increase 

Oysters

3,300 tons

2,700 tons

18% decrease 

5,250 tons

50% increase 

4,300 tons

23% increase 

Lake Condition

Average of 3

in <state> 

2

Green Water

1 foot of visibility

Mostly rough fish 

3

Brown/Green Water

3 feet of visibility

Bass and rough fish

4

Blue/Brown Water

5 feet of visibility

Mostly bass 

Permanent 

Increase in the 

Cost of Living for 

your Household 

Your Vote

Please mark one of 

the boxes to the 

right 

$0 per year 

No Further Action 

$180 per year

or $15 per month 

Program A 

$250 per year

or $20.83 per 

month

Program B 

Chesapeake Bay and Tidal Waters Lakes

10. Suppose you could choose from the following three options. Mark 

 the box at the bottom to indicate which plan you would vote for.
18

As you vote on the next questions, please remember...

 On the following pages you will see questions that present new sets of program 

options. These options require a different mix of management practices in different 

areas.

 Consider each question separately, imagining that the options are the only ones 

available to choose from.

 The next two questions cannot be compared to each other, or to the question you 

just answered.

 Do not add up effects or costs across different questions. 
19

Chesapeake Bay and Tidal Waters Lakes

11. Now suppose instead that a different set of options were available. 

 Please mark the box at the bottom to indicate which plan you 

 would vote for.

Conditions in 2025

Current 

Conditions

No Further 

Action Program A Program B

Water Clarity

Average visibility 

of 3 feet 

Average visibility of

1.5 feet

50% decrease 

Average visibility of

4 feet

33% increase 

Average visibility of

5 feet

no change 

Blue Crab

250 million crabs

235 million crabs

6% decrease 

250 million crabs

no change

280 million crabs

12% increase 

Oysters

3,300 tons

2,700 tons

18% decrease 

4,300 tons

30% increase 

5,250 tons

59% increase 

Lake Condition

Average of 3

in <state> 

2

Green Water

1 foot of visibility

Mostly rough fish 

4

Blue/Brown Water

5 feet of visibility

Mostly Bass 

5

Blue/Brown Water

8 feet of visibility

Bass and walleye

Permanent 

Increase in the 

Cost of Living for 

your Household 

Your Vote

Please mark one of 

the boxes to the 

right 

$0 per year 

No Further Action 

$60 per year

or $5 per month 

Program A 

$40 per year

or $20.83 per 

month

Program B 
20

Chesapeake Bay and Tidal Waters Lakes

12. Now suppose instead that a different set of options were available. 

 Please mark the box at the bottom to indicate which plan you 

 would vote for.

Conditions in 2025

Current 

Conditions

No Further 

Action Program A Program B

Water Clarity

Average visibility 

of 3 feet 

Average visibility of

1.5 feet

50% decrease 

Average visibility of

5 feet

83% increase 

Average visibility of

4 feet

33% increase 

Blue Crab

250 million crabs

235 million crabs

6% decrease 

280 million crabs

12% increase 

312 million crabs

25% increase 

Oysters

3,300 tons

2,700 tons

18% decrease 

5,250 tons

50% increase 

3,300 tons

no change 

Lake Condition

Average of 3

in <state> 

2

Green Water

1 foot of visibility

Mostly rough fish 

5

Blue/Brown Water

8 feet of visibility

Bass and walleye 

3

Brown/Green Water

3 feet of visibility

Bass and rough fish 

Permanent 

Increase in the 

Cost of Living for 

your Household 

Your Vote

Please mark one of 

the boxes to the 

right 

$0 per year 

No Further Action 

$60 per year

or $3.33 per month 

Program A 

$180 per year

or $15 per month

Program B 
21

13. For the choices you just made about programs, how much 

 consideration did you give to improvements in lakes compared 

 to the Chesapeake Bay tidal waters? (Please circle one.)

Considered 

Only the 

Lakes

Considered 

Lakes and 

Bay 

Waters 

Equally

Considered 

Only the 

Bay 

Waters

1 2 3 4 5 6 7

14. On a scale from 1 to 5, please rank the following based in order

 of importance when you were choosing between programs. 

Not 

Important

Somewhat 

Important

Very 

Important

Don’t 

Know

Clarity 1 2 3 4 5 DK

Striped Bass 1 2 3 4 5 DK

Blue Crab 1 2 3 4 5 DK

Lake Category 1 2 3 4 5 DK

Cost of the program 1 2 3 4 5 DK
22

15. Aside from costs, which of the following was most important

 in making your decision about which program to vote for? 

 (Please choose one.)

Recreation on the Bay

Recreation in Lakes

Preserving the environment for future generations

Availability and/or price of seafood

Value of property near the Bay

Value of property near Lakes

The possibility of visiting the Bay in the future

The possibility of visiting Lakes in the future

Other __________________
23

16. Thinking about how you voted among programs, please rate

 how much you agree or disagree with each of the following 

 statements. (Circle the number that matches your answer.)

Strongly 

Disagree Neutral Strongly Agree

My choices would have been 

different if the economy in my 

area were better. 1 2 3 4 5 6 7

It is important to improve waters 

in the Chesapeake Bay 

watershed, no matter how high 

the costs.

1 2 3 4 5 6 7

I do not think I should have to 

contribute any amount to the 

restoration of the Chesapeake 

Bay watershed.

1 2 3 4 5 6 7

I am concerned that the 

programs would hurt the 

economy.

1 2 3 4 5 6 7

It was difficult to understand the 

descriptions of the programs. 1 2 3 4 5 6 7

I believe the programs will 

improve the environment as 

described.

1 2 3 4 5 6 7

Some of the programs cost too 

much compared to what they 

would deliver. 1 2 3 4 5 6 7

The changes offered by the 

programs happen too far into the 

future for me to really care 

about.

1 2 3 4 5 6 7

The survey provided me with the 

information I needed to make a 

choice between the options 

shown.

1 2 3 4 5 6 7
24

17. If you voted for NO FURTHER ACTION in any of the choices you

 were given, please let us know how much you agree or disagree

 with each of the following statements. If not, skip to the next 

 question. (Circle the number that matches your answer.)

Strongly 

Disagree Neutral Strongly Agree

Nothing can be done to 

improve the quality of the Bay 

and freshwater lakes.

1 2 3 4 5 6 7

I would not vote for any 

program even if there were no 

costs to my household.

1 2 3 4 5 6 7

I voted for NO FURTHER 

ACTION because the costs were 

too high compared to the 

improvements.

1 2 3 4 5 6 7

I voted for NO FURTHER 

ACTION because I am against 

more government requirements 

and spending.

1 2 3 4 5 6 7
25

Strongly 

Disagree Neutral Strongly Agree

I voted for a program because I 

thought it would also lead to 

better water quality closer to 

my home. 

1 2 3 4 5 6 7

I voted for a program more for 

future generations than for 

myself. 

1 2 3 4 5 6 7

I voted for the program 

because I may visit the 

Chesapeake Bay or lakes in the 

watershed in the next few 

years. 

1 2 3 4 5 6 7

I voted for a program because I 

believe it is our responsibility to 

preserve wildlife and habitat in 

the Bay and Lakes. 

1 2 3 4 5 6 7

I voted for a program so others 

could enjoy the Bay and lakes, 

rather than for myself. 

1 2 3 4 5 6 7

18. If you voted for Program A or Program B in any of the choices

 you were given, please let us know how much you agree or 

 disagree with each of the following statements. If not, skip this 

 question. (Circle the number that matches your answer.)
26

19. Are you a male or female?

Questions about you and your household

Finally, we would like to ask you a few questions about you and your household for 

statistical purposes. The reports prepared for this study will summarize findings 

across all responses and will not associate responses with an individual. Your 

answers will not be saved or stored in a way that can be associated with your name 

or address. You will not be contacted about your responses or this survey. 

Male

Female

20. What is your age?

______ years old

21. What is your current marital status?

Under $25,000

$25,000-$34,999

$35,000-$49,999

$50,000-$74,999

$75,000-$99,999

$100,000-$199,999

$200,000 or more

22. How many children under age 18 are living at your home?

______ children

23. In 2011, what was your total pre-tax household income, including

 all earners in your household?

Single, never married

Married or living with a long-term partner

Separated or divorced

Widowed
27

25. Which of the following best describes the home or apartment you live in?

Owned by you or someone in your household with a mortgage or loan

Owned by you or someone in your household without a mortgage or loan

Rented

Other: _____________________________________________

24. What is the highest degree or level of school you have completed?

No high school diploma

High school diploma or GED

Some college credit or college degree

Some graduate school or professional school credit or a graduate

or professional degree

26. Have you or any member of your family ever worked for any of the

 following industries or jobs? (Please fill in the box next to all that apply.)

Agriculture

Commercial fishing

Tour guide for fishing

No one in my family has worked in these industries

Thank you very much 

for your help.
The public reporting and recordkeeping burden for this collection of information is estimated 

to average 2 hours per response. Send comments on the Agency's need for this 

information, the accuracy of the provided burden estimates, and any suggested methods for 

minimizing respondent burden, including through the use of automated collection techniques 

to the Director, Collection Strategies Division, U.S. Environmental Protection Agency 

(2822T), 1200 Pennsylvania Ave., NW, Washington, D.C. 20460. Include the OMB control 

number in any correspondence. Do not send the materials to this address.
Attachment E. Focus Group and Cognitive Interview Summary

This report summarizes the main findings from the ten focus groups and three sets of cognitive interviews

conducted as part of the design and pre-testing of the Chesapeake Bay Watershed stated preference

survey.

The ten focus groups took place in the following locations on these dates:

1. Baltimore, MD 1/12/12

2. Baltimore, MD 1/23/12

3. Richmond, VA 2/01/12

4. Richmond, VA 2/02/12

5. Washington, DC 2/29/12

6. Washington, DC 3/01/12

7. Harrisburg, PA 3/26/12

8. Harrisburg, PA 3/27/12

9. Raleigh, NC 4/16/12

10. Raleigh, NC 4/17/12

Each focus group was composed of seven to ten individuals that were randomly selected by the focus

group facilities. Participants were eligible if they were eighteen years of age or older, if they were not

full-time students, and if they had not participated in a survey or focus group in the last six months. A

college degree was a requirement for some focus groups, but not for others. Each of the focus groups

included individuals with diverse socioeconomic backgrounds, based on characteristics such as income,

marital status, age, race, education, occupation, and gender. Participants were compensated for their

participation in the focus groups. Each focus group session was approximately two hours long. All

sessions were video recorded.

Cognitive interviews took place in the following locations on these dates:

1. Washington, DC (10 interviews) 5/03/12

2. Richmond, VA (8 interviews) 5/08/12

3. Harrisburg, PA (8 interviews) 5/11/12

Interviews were conducted one-on-one primarily using a think aloud technique. Interview subjects were

randomly selected by the focus group facilities using the same eligibility rules applied to the focus

groups. All interviews were audio recorded.

Between each focus group and set of interviews the survey instruments were revised based on feedback

from respondents. Detailed summaries of each focus group and set of interviews are available as separate

reports.

Focus Groups Overview

Focus Groups 1 and 2 (Baltimore, MD)
Focus Group 1 was conducted in Baltimore, MD and focused on four main objectives:

1. Identifying popular recreation areas near the Bay

2. Identifying aspects of the natural environment in or around the Bay itself that are important to the

participants

3. Discussing the best way to communicate levels of different aspects of the Chesapeake Bay to

respondents (e.g., fish populations)

4. Exploring payment vehicles.

For recreational purposes, respondents identified cleanliness of waters and presence of wildlife and

healthy vegetation as important elements of the natural Bay. Outside of recreational opportunities, aquatic

life (e.g., population, subaquatic vegetation) was also important. Respondents in this group did not

exhibit strong preferences for particular species, but thought in terms of broad categories such as fish and

shellfish, and preferred to think about these in terms of changes in absolute number of fish, percent

differences, and total stock size. Respondents also indicated that water quality was important to them as

an input for plants and animals, but also as something they care about directly. Respondents focused on

clarity as a measure of quality, and did not necessarily care about algae blooms, dead zones, and red tides,

in part because these terms were unfamiliar and too technical.

The discussion on potential payment vehicles revealed that people believed they were already paying for

Chesapeake Bay restoration efforts through separate assessments or as additions to their water bills.

There was some indication of willingness to pay additional amounts for Bay improvements and

suggestions included taxes and recreational fees.

Focus Group 2, also in Baltimore, pursued the first three questions from Focus Group 1 in addition to

responses on some basic choice questions.

Participants were provided with a detailed listing of Bay environmental attributes and asked to select

those important to them or to add their own. “Suitable for aquatic life” was the most widely appreciated

attribute, but this was largely because the term was considered all-encompassing. Several species were

identified as important, including rockfish, blue fish, crab, oysters, and bird species. Clarity was

identified as an aspect of the environment that was important, but was not as prominent as a key indicator

as it was in Focus Group 1; some participants noted that they expect the water in the Bay to be murky

given the nature of the water body. There were no preferred metrics for providing information on

changes in the attributes.

When presented with some basic choice questions the group and participants were generally willing and

able to consider tradeoffs of money for changes in different attributes. They seemed particularly

concerned about stalling a worsening baseline (i.e., preventing further declines in environmental quality

in the future). More information on the timing of payments and improvements is needed to pursue these

tradeoffs in more detail.
Focus Groups 3 and 4 (Richmond, VA)

Focus Group 3 was conducted in Richmond, VA. This focus group pursued the same goals as in the

Baltimore focus group 2, in part to assess regional variation in important endpoints or aspects of the

natural environment. Thus, the main goals of the focus group were to:

1. Identify popular recreation areas near the Bay

2. Identify aspects of the natural environment in or around the Bay itself that are important to the

participants

3. Determine the best way to communicate levels of different aspects of the Chesapeake Bay (i.e.,

endpoints)

4. Obtain reactions to simple choice questions.

Respondents here identified many of the same important attributes as those in the Baltimore focus groups,

with a particular emphasis on the water being safe for humans and wildlife, clarity, the ability to support

aquatic life, and the number of fish and shellfish (expressed as crabs, oysters, or both). Other aspects

raised by respondents included “clean beaches,” nice views, and that the Bay be “clean and natural.”

Most respondents acknowledged a preference for improvements to the rivers, in particular the James

River, due to the proximity to where they live and recreate. Many respondents noted a potential tradeoff

between benefits from industry and recreational uses along the Bay. Respondents also suggested that

improvements to the Bay would need to be the result of concerted effort across states because so many

rivers enter the Bay.

Focus Group 4 was also conducted in Richmond. The main goals of the focus group were to:

1. Assess the effectiveness and clarity of draft text with background information on the state of the

Bay

2. Assess reactions to simple choice questions

Participants were asked to read a description of the state of the Bay, and then were presented with a set of

structured choice questions with varying attributes, but no costs.

Reactions to the background text suggested that a simpler depiction of the ecological production function

of the Chesapeake Bay and related waters was necessary. Some terms were difficult for participants to

understand (e.g., “deep water refuge”) and many participants did not make key connections between

precursors affected by policy (e.g., sediment) and environmental consequences (e.g., clarity). Generally,

respondents wanted clearer information about sources of pollutants and their effects on the environment.

As presented, respondents generally had difficulty trading off among attributes. This could be in part

because a cost attribute was not included in these preliminary choice questions, or perhaps because the

focus group started with an overview of the entire Bay, rather than eliciting and identifying attributes

individuals care about. The need for a more solid ecological production function was apparent:

participants had a difficult time thinking of the fish as anything but food, and some found the inclusion of

clarity confusing as its importance to the ecosystem had not been established. However, there was some

indication that clarity was a good proxy for recreational endpoints. Finally, respondents expressed a

desire for an index to describe overall Bay health.
Focus Groups 5 and 6 (Washington, DC)

Focus Group 5 was conducted in Washington, DC. The main goal of this focus group was to compare

two versions of a draft survey instrument that differed by whether the quality of the Bay was described in

terms of attributes that are ecological “inputs” (dissolved oxygen, sub-aquatic vegetation, clarity) or by

endpoints or ecological outputs or “endpoints” (water quality score, game fish, crabs, oysters). Several

alternative formats and representations of the choice questions were also presented to respondents.

Most participants preferred the endpoint attributes because they were more tangible and closer to things

they care about directly. Some respondents wanted input attributes along with these endpoints.

Generally, it wasn’t clear whether participants cared about these attributes independently or whether they

served together to define overall quality of the Bay. Many liked the use of a water quality score as an

overall measure, but wanted more information to make it more tangible.

There was some confusion about how input attributes were defined and what they were really capturing.

Some discussion suggested that attribute levels were not being considered independently, e.g., “dissolved

oxygen helps sub-aquatic vegetation therefore care about dissolved oxygen” (but both are independently

represented in choice questions). This suggested that if inputs were to be used in the survey, the

ecological production function (i.e., the link between the inputs and the endpoints) would need to be more

carefully described.

Other issues raised during the focus group included tourism and economic impacts; how payments would

be allocated among businesses, farms, individuals, and states; and potential “upstream benefits” beyond

the Chesapeake Bay itself.

Focus Group 6 was also conducted in Washington, DC. The primary goals were to continue testing

“input version” and an “endpoint version” of the survey, and to evaluate alternative presentations of

choice scenarios.

Most participants preferred the endpoint survey because they could relate to the attributes better and

because the information treatment for the inputs version was too technical. Some respondents wanted the

more technical information, however. This suggested that a unified information treatment, compromising

between the levels of background and detail in these two instruments, was desirable.

Other important findings from the focus group included a preference for arrows indicating the direction of

a change in attribute levels, rather than a graph showing changes over time; willingness to pay amounts

almost always exceeded the $135/year maximum in the choice questions; and people found it useful to

have both monthly and annual costs.

Focus Groups 7 and 8 (Harrisburg, PA)

Focus Group 7 was conducted in Harrisburg, PA largely to assess how respondents farther out in the

watershed would react to the survey. Other goals included testing a figure to depict the ecological
production function, testing different presentations of water quality indicators, evaluating the inclusion of

an “upstream” attribute to reflect environmental improvements beyond the Bay tidal waters, and testing a

payment vehicle.

Participants were first given the “input version” of the survey and then were given the “endpoint version.”

In both versions, participants generally found the ecological production function figure redundant with

what was provided in other parts of the survey, and did not react well to specific graphics within it.

There was an approximately even split between preference for the inputs (dissolved oxygen, water clarity,

aquatic grasses) vs. the endpoints (water quality scores, striped bass, blue crab, oysters) versions of the

survey. Respondents also believed a third version of the survey that included only water quality indices

did not provide enough information for making choices.

Participants reacted well to inclusion of the freshwater attributes. Several focused first on that when

answering choice questions, although there was general agreement that doing something for the Bay was

important. Some participants were unable to separate upstream attributes from Bay attributes, suggesting

the survey needs to better specify how these may change independently.

All participants selected a program rather than the status quo at the costs in the survey drafts. It was still

difficult to conclude whether participants held values for the individual attributes and considered tradeoffs

among them, or were interested in improving water quality in general and seemed to just pick the program

that cost the least.

Participants accepted the payment vehicle as a permanent cost-of-living increase, and none admitted that

they thought they could escape the costs. Two respondents indicated that the permanent nature of the

payment figured prominently in their choices, but suggested that any time period of 10+ years might as

well be permanent.

Focus Group 8 was also held in Harrisbug, PA with the same goals as Focus Group 7. The order of the

survey versions was reversed.

Reactions to the survey versions were similar to Focus Group 7, with a split between input and endpoint

versions of the survey (five to three). Framing attributes in terms of the percent of water segments in

compliance with standards seemed to help respondents put the attribute levels in perspective. They also

liked that the input version included a diversified set of attribute metrics, and found that attributes

expressed as feet and acres were especially easy to relate to.

Participants seemed willing to pay money to improve the Bay. They did not question why they have to

pay for the cleanup, which weakly suggests an acceptance that they contribute to the degradation, or at

least bear some responsibility towards its improvement.

Respondents generally accepted the inclusion of a freshwater upstream attribute(s). It was unclear

whether they necessarily favored upstream water quality more than in the Bay. This was surprising given

Pennsylvania’s abundance of small streams and lakes, and the fact that the State is not adjacent to the

Bay. On the other hand, most respondents seemed to be nonusers of both the Bay and upstream waters.
The independence of freshwater and bay water quality attributes needs to be further emphasized in the

text.

Respondents seemed to value individual attributes, but it was unclear whether they necessarily hold

shadow values for the individual attributes. In other words, in some cases respondents may simply have

been focusing on one attribute or another, but in fact valuing the same thing. For example, some people

valued oysters because they filter the water, so essentially the good they valued was water quality or

overall Bay health (which is redundant with other attributes, such as the water quality score). This

suggested that the independence of attributes must be further emphasized in the text, and any text

promoting perceptions of dependence among attributes should be reduced.

Lastly, we also experimented with a payment vehicle consisting of five years of payment going into a

fund to support the program indefinitely. This payment vehicle did not work well, and led to many open

questions. Participants also did not believe fees under this payment vehicle would actually end after 5

years.

Focus Groups 9 and 10 (Raleigh, NC)

Focus Group 9 was held in Raleigh, NC in part to test for the salience of TMDL-related improvements in

a state that is nearby, but is not part of the Chesapeake watershed. Other goals included testing a unified

information treatment, and evaluating whether a cost-of-living payment vehicle would work beyond

watershed states. Again, an “input version” and an “endpoint version” were tested.

The group was primarily non-users, but participants seemed to understand the map and scope of the

Chesapeake Bay and watershed. Responses were generally positive for the unified information treatment,

including the figure on the ecological production function.

Participants accepted the payment vehicle as realistic and feasible. There was some initial confusion from

a few participants as to whether they should be responding as themselves, or as someone who lives within

the watershed. After clarifying this, participants accepted the payment vehicle, although some thought

the presentation should be simplified and that it should be clearer on which costs occur to those inside

versus outside the watershed.

Participants were able to complete both versions of the survey, but all preferred the endpoint version

because it was more salient; it was not clear to them how inputs related to potential endpoints they care

about.

Part of the preference for the endpoint version also appeared to be presentational. Several participants

noted that the small graphics for that attributes (line drawings of a fish, crab, and oysters) made it more

appealing. Also, the use of indicator arrows for increases and decreases was preferred to the inputs

versions that did not contain these indicators. This suggested a need to make the versions visually

commensurate.
Everyone chose one of the two improvement programs, although most would never or rarely see use

benefits. Few respondents chose the status quo at the costs presented. However, when program costs

exceeded $100 per year there was a shift away from that alternative even when it offers larger benefits,

suggesting price sensitivity at levels above this.

Focus Group 10 was held in Raleigh, NC with many of the same objectives as Focus Group 9. We also

wanted to test whether respondents would accept choice questions where environmental quality was held

constant in 2025 under the status quo or “No Further Action” option.

This group, for the most part, consisted of nonusers, some of which had not even heard of the Chesapeake

Bay. The distinction between the Bay (“tidal waters”) and upstream confused some, suggesting we

needed to emphasize this more clearly in the text.

Respondents seemed accepting of both versions of the survey, with a slight preference towards the

“endpoint” version. Respondents found water quality scores to be difficult to comprehend and evaluate,

although the numbers were easier to understand.

Participants stated they are willing to pay for improvements to the Chesapeake Bay watershed. This

included nonusers whose WTP reflected bequest and existence values. Somewhat higher cost options for

the instruments in this focus group increased “no further action” responses.

Participants seemed to accept the general cost of living payment vehicle, but the information needed to be

presented more clearly and succinctly with a greater emphasis on what costs occur within versus outside

of the watershed. Similarly, respondents appeared to accept a constant baseline scenario (i.e., no

projected decline in environmental attributes in the absence of policy), but wanted a clear justification and

information to support this, such as examples of current programs that are already in place.

Cognitive Interviews Overview

Cognitive Interview Set 1 (Washington, DC)

This set of interviews tested input and endpoint versions of the survey, with constant and declining

baselines. For all sets of interviews respondents were only given one version of the survey. Due to new

information on upstream modeling limitations, the upstream attribute was now presented as a Lake

Condition attribute (replacing the upstream water quality index). This had the advantage of being more

concrete than the water quality index score, which prior focus groups had found difficult to understand.

The Lake Condition attribute refers to the trophic state of a lake and is described on the survey using

visibility, water color, and dominant types of fish. The attribute was received well by the subjects and

seemed to work fine in the choice questions.
A potential problem that came up in multiple interviews was that on the distinction between the parts of

the watershed (tidal vs. upstream) was sometimes forgotten by the time respondents came to the choice

questions. This suggested a need to better clarify this distinction so that it “sticks” throughout the survey.

There were two different problems with the payment vehicle. One respondent did not understand why her

water bill would increase and another confused the “cost of living” increase with general inflation, and

assumed salaries would increase to offset it.

The endpoint version of the survey worked well with respondents, who were able to evaluate the attribute

levels, tradeoffs, and make an economic decision.

Most subjects responded well to the input version, accepting the attributes and evaluating program

options based on them. One respondent had difficulty and wanted to have information on endpoints

rather than inputs. He answered the questions, but based responses largely upon his general knowledge of

water quality in the area rather than specific input attribute levels.

Several respondents had suggestions for wording improvements and clarity, and some felt the survey was

long. At the same time, however, some wanted more information.

Aside from these issues, the interviews went relatively smoothly. Most subjects were able to follow the

information treatment, accepted the payment vehicle, evaluated attribute levels, and considered tradeoffs

between attributes when answering the choice questions. Several respondents indicated they would

choose the “No Action” status quo based on an evaluation of the cost and the described improvements.

Cognitive Interview Set 2 (Richmond, VA)

Four subjects were given the endpoint version of the survey with a worsening baseline. All subjects

accepted the scenarios and payment vehicle. Throughout the survey we relabeled the “Upstream

sections” of the bay as “Lakes and rivers,” which eliminated confusion regarding the two regions of the

watershed. Providing an example question prior to the choice scenarios helped to familiarize the subjects

with the question layout before they were asked to make a choice.

Four subjects were given the input version of the survey with a worsening baseline. All subjects accepted

the scenarios and payment vehicle, but one needed some additional statements that all else was the same

across these options. She was mainly concerned about equity in costs and had concerns about ancillary

benefits and costs pertaining to how the program would be implemented, and who would regulate it.

Overall, subjects were careful to analyze trade-offs among the attributes and costs across the different

alternatives.

Most commented on the length of the survey but acknowledged that all the material is necessary to

answer the choice experiment questions. Some suggested a strong cover letter is needed to emphasize

how important it is to complete the survey.
Cognitive Interview Set 3 (Harrisbug, PA)

Four subjects were given the input version of the survey with a declining baseline. Most subjects

accepted the scenarios and payment vehicle, but one found the choice questions difficult and confusing

and ultimately did not make decisions based on attribute levels. Generally, respondents were able to

answer the program choice questions, although the description of the dissolved oxygen criterion was

difficult for two subjects. One subject didn’t fully understand the goals, while a second was concerned

that changes in dissolved oxygen involved a chemical process with ambiguous risks. This suggested a

need to clarify the description of the dissolved oxygen attribute.

Because the survey instrument had worked well in prior interviews, we probed deeper on some particular

issues on how respondents approached the choice questions. For the input version of the survey,

respondents seemed to first look at the cost of the programs to see if any were affordable and then tended

to support the lower cost option. Some respondents noted lake condition and clarity as key factors in their

decisions, but did not exhibit strong preferences across the other attributes. These two observations are

not so surprising given that most participants had never visited the Bay. Nonusers may have positive

WTP without strong preferences over attributes.

Similar results were obtained from four interviews using the endpoint version of the survey with a

declining baseline. All of the subjects accepted the scenarios and payment vehicle, but one had difficulty

understanding why conditions for the other Bay attributes (fish and shellfish) wouldn’t necessarily

increase if the Water Quality score (capturing improvements in clarity and DO) improved. (The Water

Quality score attribute was ultimately dropped in favor of clarity, which seemed more salient to most

respondents.)

Most subjects first looked at the cost of the programs to see what they could afford then proceeded to look

at the other attributes provided. Lake condition seemed to be the driver for those that selected programs

coupled with the desire to see some increase among the fish populations. Subjects did not have strong

preferences about which species they preferred, but choices were sensitive to large improvements in the

attributes. Water quality conditions in the Chesapeake Bay did not figure prominently in most subjects’

decision making, although one subject noted that this was the most important of them all.

Most respondents suggested that it would be useful to have information early in the instrument to describe

what was going to be asked later in the survey.

Additionally, some location information wasn’t fully clear, including where the Susquehanna and key

cities were located on the map and whether there were any nearby lakes. A more detailed map was

suggested.