Valuation of Ecological Benefits: Improving the Science 

Behind Policy Decisions 

PROCEEDINGS OF 

SESSION VI: METHODOLOGICAL ADVANCES IN STATED PREFERENCE 

VALUATION 

A WORKSHOP SPONSORED BY THE U.S. ENVIRONMENTAL PROTECTION 

AGENCY’S NATIONAL CENTER FOR ENVIRONMENTAL ECONOMICS (NCEE) 

AND NATIONAL CENTER FOR ENVIRONMENTAL RESEARCH (NCER) 

October 26-27, 2004 

Wyndham Washington Hotel 

Washington, DC 

Prepared by Alpha-Gamma Technologies, Inc. 

4700 Falls of Neuse Road, Suite 350, Raleigh, NC 27609 
ii

ACKNOWLEDGEMENTS

This report has been prepared by Alpha-Gamma Technologies, Inc. with funding from 

the National Center for Environmental Economics (NCEE). Alpha-Gamma wishes to 

thank NCEE’s Cynthia Morgan and the Project Officer, Ronald Wiley, for their guidance 

and assistance throughout this project. 

DISCLAIMER

These proceedings are being distributed in the interest of increasing public understanding 

and knowledge of the issues discussed at the workshop and have been prepared 

independently of the workshop. Although the proceedings have been funded in part by 

the United States Environmental Protection Agency under Contract No. 68-W-01-055 to 

Alpha-Gamma Technologies, Inc., the contents of this document may not necessarily 

reflect the views of the Agency and no official endorsement should be inferred. 
iii

TABLE OF CONTENTS 

Session VI: Methodological Advances in Stated Preference Valuation

 Embedding in the Stated-Preference Methods 

 Michael Hanemann, University of California - Berkeley; Jeff Lazo, 

National Center for Atmospheric Research ............................................................1 

 Experimental Tests of Provisions Rules in Conjoint Analysis for 

Environmental Valuation 

 Laura Taylor, Georgia State University; Kevin Boyle, University of 

Maine; Mark Morrison, Charles Sturt University ...................................................6 

 Internet-Based Stated Choice Experiments in Ecosystem Mitigation: 

Methods to Control Decision Heuristics and Biases 

 John P. Hoehn, Frank Lupi, Michael D. Kaplowitz, Michigan State 

University.................................................................................................................8 

 Discussant 

 Joseph Cooper, USDA, Economic Research Service ............................................44 

 Discussant 

 Thomas H. Stevens, University of Massachusetts - Amherst ...............................51 

 Summary of Q&A Discussion Following Session VI ..........................................57 
1

Embedding in the Stated-Preference Methods

By 

Michael Hanemann, University of California, Berkeley 

Jeff Lazo, National Center for Atmospheric Research 

(Summary of Dr. Hanemann’s presentation)

Dr. Hanemann began by stating that the reason he and his colleague focused their 

attention on embedding is that he believes “embedding is one of the most serious issues 

that remains not satisfactorily resolved in contingent evaluation, and it’s a major focus of 

the critics.” He continued, “This research wants to try and come to grips with what is 

causing embedding, whether these causes operate in market valuation as opposed to nonmarket valuation, whether they occur in stated preference based on conjoint analysis, and 

lastly, what can be done to deal with embedding in stated preference generally.” 

Stating that different terminologies are often used, Dr. Hanemann went on to clarify the 

meaning of the term “embedding” as he uses it. He stated that embedding involves three 

elements: (1) insensitivity to scope, so that a larger item is not valued more than a smaller 

item; (2) sub-additivity, meaning that the value of a set of items is less than the sum of 

the values of the items individually, and (3) order effects—the order in which an item is 

valued affects its value. He said that he believes “one can write down utility functions 

which explain all three effects in terms of diminishing marginal rate of substitution, 

income effects, and substitution effects.” 

Referring to a formula with variables representing public goods and income, Dr. 

Hanemann said he thinks “something like this can represent mental accounting; that is, 

mental accounting can be expressed as a form of utility function. He continued, “What I 

want to stress here is that I don’t think the economic structure of preferences is all that’s 

going on with embedding, and I want to focus on other features, such as features of the 

questionnaire and features of the elicitation format, but also, more basically, features of 

how people think and make judgments about items.” Dr. Hanemann explained that the 

methodology of the research is to replicate some of the existing studies in the literature, 

using the same sort of setting and the same survey mode while at the same time adding 

features to the survey which are designed to explore some of the hypotheses that focus on 

the three items he identified. 

Starting off by looking at scope effects, Dr. Hanemann asked, “Why might somebody 

give you the same willingness to pay for a larger item as for a smaller item?” He 

suggested five possible explanations: (1) The survey is flawed and doesn’t really capture 

what the person feels. (2) The person doesn’t see the larger item as any better than the 

smaller item. (3) The person feels that if he pays for the smaller item, he actually gets the 

effect of the larger item anyways, so there’s no point in offering more money. (4) The 

person feels that the larger item isn’t feasible, and therefore pointless. (5) The person 

thinks that the larger item actually only costs the same. He stated, “To explore these 

explanations, one needs to incorporate what are called manipulation checks, that is, 
2

questions in the survey or “think-alouds” or verbal protocols to get at what the respondent 

was thinking of. We focus on monetary valuation and get anomalous results, and we feel 

that’s because of a flawed elicitation of monetary valuation. However, it may not be 

flawed—it may simply be that these other things are going on, but we don’t normally 

look for them. So, the focus of the research and the replication that I’m conducting is to 

investigate these other explanations. Of 60 or 80 scope studies, there are only 4 or 5 that 

I’ve seen that do this. A very nice recent one is coming out shortly in the Journal of 

Environmental Economics and Management by Heberlein, Bishop, and Schaefer. They 

express that conventional economists look at economic scope, what they call an affective 

scope and cognitive scope, and they get at these things by asking a series of questions—

this is what I was referring to as manipulation checks. For instance, in a question to 

evaluate wolves, they ask, “How important are wolves to you, personally? –Not at all 

important—Somewhat important—Etc.” Or: “In valuing a population of 800 wolves 

versus a population of 300 wolves in northern Wisconsin, how would you rate a 

population of 800 wolves? –Extremely bad—Somewhat bad—Bad—Neither bad nor 

good—Etc.” 

Dr. Hanemann revealed that “what they find when they use these manipulation checks is 

that they line up with the monetary valuation. When respondents like the larger item 

more, they give it a higher value. Sometimes the respondents like the smaller item more, 

and then they value that item accordingly compared to the larger item.” Providing 

another example, which he said “is not widely reported in the study that Bill Desvouges, 

Kevin Boyle, et al. did on birds” Dr. Hanemann said that there was actually a 

manipulation check in the survey. He stated, “Remember, the focus was on covering 

waste oil holding ponds on the flyway to protect birds from being killed—2,000 birds—

20,000 birds—200,000 birds. The researchers posed the question: Covering waste oil 

ponds will not significantly affect populations—Strongly agree?—Agree?—Neither 

agree nor disagree?—Etc. There was the same sentiment. That is, most people felt that 

this didn’t make a big difference. So, it’s not surprising to me that they then found no 

difference in monetary value between those items.” 

Moving on to the issue of feasibility, Dr. Hanemann cited Baruch Fischoff’s paper in 

which he looked at willingness to pay for pollution cleanup along variable segments of 

the Susquehanna River. Dr. Hanemann focused on some debriefing questions that were 

not used in the study’s data analysis but that were reported. Fischoff found that in a postsurvey phone interview people remembered poorly how many miles of the river were to 

be cleaned up, but the people who thought there were more miles had a higher 

willingness to pay. Dr. Hanemann summarized, “In other words, some of the noise in the 

willingness to pay responses seems to correlate with noise in what the size of the 

commodity was. Also, a significant fraction of people didn’t think that a thousand miles 

could be cleaned up, and again that appears to have influenced their responses.” 

Dr. Hanemann went on to note that “most of these studies use the open-ended format—

how much are you willing to pay?—and that introduces additional complications because 

in addition to valuing the item, people don’t want to pay more than their fair share and 

they don’t want to pay more than the item costs. In this context, one issue is maybe a 
3

larger item will involve more people, and the cost per household for the larger item may 

not be any greater than the cost per household for the smaller item.” He cited some 

evidence from a phone survey he had done in Ohio regarding river cleanup in which he 

found that willingness to pay is correlated with cost, and cost doesn’t vary with scope. 

He said, “What we are finding is that things that ought to correlate, rationally, with the 

willingness to pay also don’t vary with scope. So, the focus of this research is to 

replicate some of the scope studies but to add these questions to see if this holds in some 

of those studies.” 

Saying he wanted to relate all of this to the literature on market research, Dr. Hanemann 

cited “a very interesting series of papers by Chris Hsee at Chicago, who has worked on 

what he calls joint versus separate evaluation.” He said that in implementing the strategy, 

you first describe a market goods item to someone and then ask, “How much would you 

pay for this?” Another group of people is asked the same question regarding a different 

item. A third group is then asked to evaluate the two items together, so you achieve both 

separate and joint evaluation. Dr. Hanemann said that Hsee’s premise is that “assessing 

an item in isolation is more difficult than assessing two or more items together, and 

because of this difficulty, people adopt different response or judgment strategies in 

assessing a single item in separate evaluation than in joint evaluation.” He stated that 

Hsee frames his comparison “in terms of evaluability: When assessing an item in 

isolation, the judgment is influenced more by attributes that are easy to evaluate, even if 

they are less important than other attributes which are hard to evaluate.” However, when 

people assess two or more items together, it is easier to compare the attributes—one 

against the other—and more weight is placed on the more important attributes. Through 

your choice of things, you can therefore switch the ranking of the choice of items. 

To illustrate the point, Dr. Hanemann cited one of Hsee’s examples concerning the 

purchase of a music dictionary as a gift for a friend. Given the choice between a 

dictionary with 20,000 entries that has a torn cover and a dictionary in perfect condition 

but with only 10,000 entries, he revealed that in isolation people chose the smaller 

dictionary more often, but when they used joint evaluation people chose the larger 

dictionary more often and tended to overlook the blemish. Dr. Hanemann said that 

studies from the environmental literature show the same thing. He went on to reiterate 

the widely recognized “greater difficulty of doing separate evaluation as opposed to joint 

evaluation.” 

Dr. Hanemann said Hsee also points out “the link to another concept in psychological 

theory called norm theory: when evaluating an item in a separate evaluation, people 

think about the larger category to which they think the item belongs and then they 

compare it to the norm for that category. In joint evaluation they compare the two items 

as opposed to comparing each item with the norm from an imagined category.” He said 

he stresses this point because he thinks that “all cognition is relative, not absolute, and 

the norm theory suggests that if I don’t give you a standard of comparison but instead just 

ask you to evaluate a single item, you invent a standard of comparison.” This ends up 

being “more noisy” because it’s not controlled by the researcher. 
4

“Here are the implications. I see this as essentially the same phenomenon as what some 

have termed coherent arbitrariness. As I said, cognition is relative to something—to a 

norm—and that something affects the evaluation. It makes evaluation in isolation noisy, 

or arbitrary, but I think it’s arbitrary within some range.” Dr. Hanemann said the 

important thing, he believes, is that “this applies not just to the monetary evaluation 

(expressed willingness to pay) but to all dimensions of liking and judgment for an item. 

So, again, the aim is to test this by replicating some studies. I also think mathematically 

you would say that separate evaluation involves an element of noise, and then there’s less 

noise or less uncertainty when a second item is considered, and so one could write down 

some mathematical formalisms.” 

Dr. Hanemann continued, “This also relates to some recent work by Ian Bateman that 

was published this year looking at order effects—looking at the evaluation of multiple 

items and comparing what he calls step-wise versus advanced-disclosure designs.” He 

explained, “With step-wise you get to see one item and you are asked to evaluate that. 

Then you get to see another item, a subset or something—then you get to see another 

item—but each time you value an item before knowing what else is coming. With full 

disclosure, on the other hand, you’re shown everything—it’s laid out before any of the 

valuation questions are asked.” Dr. Hanemann stated, “What Ian (Bateman) finds is that 

the order effects appear very regularly with the step-wise design and go away with the 

advanced disclosure. How does this relate? I think this is like the distinction between 

separate evaluation and joint evaluation—it’s the same type of phenomenon. It’s not 

limited in any way to non-market goods, and it’s not a feature of monetary evaluation as 

opposed to other dimensions of evaluation, and I think it moves things around much more 

than the economic formalisms of income effects and substitution effects.” 

“So, what I’m doing is replicating Ian Bateman’s work, comparing step-wise with 

advanced disclosure, but also measuring not just monetary evaluation but other 

dimensions of liking and valuation for the goods in non-monetary terms and seeing if 

they have order effects in one treatment and not in the other treatment.” 

“If there is a difference between separate evaluation, thinking of items in isolation, versus 

thinking of them together, the question arises: Which is better?” Dr. Hanemann stated 

that “the NOAA panel argued strongly for separate evaluation for external tests of scope, 

not internal tests of scope.” He added, “One reading of Hsee’s work is that the external 

tests of scope, the separate, are much more noisy. They’re rooted less securely. In a 

sense, they go against how human cognition works, and in fact, what happens is a person 

invents something with which to contrast or compare the item being evaluated.” 

Dr. Hanemann noted that “Hsee says in a recent paper that this also explains the 

difference that’s been observed between predicted utility and experienced utility, because 

when I ask you to predict your behavior or your choice, that’s like a joint evaluation 

because you imagine several outcomes and you compare them. But what actually 

happens in life is you choose one of them (or one of them gets chosen) and you 

experience that—you decide to move to the West Coast, for example—and then three 

years later you’re asked how did you like it. That’s more like separate evaluation; you 
5

experienced one thing and you didn’t experience anything else, and he argues that 

separate evaluation may be more realistic for assessing actual experienced utility.” 

Dr. Hanemann continued, “My inclination is to prefer joint evaluation, but—this is the 

point I want to emphasize—it seems to me that joint evaluation is susceptible to the same 

underlying forces. Now, I just said that in joint evaluation you have contrast, but the 

contrast is influenced by the particular items that are involved in that contrast. If you had 

other items the assessment might have come out differently, and there is in fact a large 

literature in market research on, for example, the number of alternatives, the variety of 

alternatives, the number of different attributes or dimensions on each alternative, a range 

of values.” He went on to offer this example: “Joel (Huber) and a colleague did a 

beautiful work on decoy effects in asymmetric dominance. You choose between A and 

B. Then I add an item C, which is actually dominated by A or B but shifts your choice 

between A or B because it makes one of the items look better. Why? Because you’re 

evaluating attributes of a particular item relative to the range of attributes in the choice 

set. It’s the same sort of cognitive imperatives potentially affecting joint evaluation, and 

Ian (Bateman) has a CV study showing the presence of decoy effects in multi-item 

evaluation.” 

In closing, Dr. Hanemann said, “I want to end with two points. First, one of the 

criticisms of non-market valuation relative to market valuation is that you tell me you 

would pay so much for this particular item, but there are other items out there—there are 

other brands, etc. How much would you pay for the other items and does it add up? A 

major difference with public goods is there’s not a set of other items. If I ask you to 

value a particular flavor and brand of yogurt, we both know you can walk down the road 

to the supermarket where there’s a whole shelf of other brands of yogurt. If I ask you to 

value a program to protect frogs in a particular part of Rhode Island—well, it’s true that 

you could have programs to protect other creatures in other parts of New England, but the 

point is there’s no reason to believe that anybody’s planning to do anything about other 

creatures in Massachusetts or any place else—it’s not obvious that these other public 

goods are out there. . . . So, one issue is what people think are the other items when you 

ask them about a public good. With market goods, you know what’s in the 

supermarket—you don’t need to ask them.” 

“I’m anticipating possible conclusions, but I haven’t reached them. The surveys that 

we’re doing now are meant to add features which test the speculations I’ve told you 

about. If they come out, this might be an assessment of embedding effects—that in some 

sense, joint evaluation with advanced disclosure is preferred, and that means that order 

effects can be controlled, and I think the scoping sensitivity gets controlled also and 

becomes less of an issue. The remaining issue, which is incorrigible, is that in any sort of 

joint evaluation the results are sensitive to the set of designs, and we need to think of 

ways of standardizing this or controlling this—we can’t escape it, but we could perhaps 

agree on a protocol for doing this so we bring this effect under control.” 

“Thank you.” 
6

EPA Agreement Number: R830820

Title: Experimental Tests of Provisions Rules in Conjoint Analysis for Environmental

Valuation

Investigators: Laura Taylor, Mark Morrison and Kevin Boyle

Institution: Georgia State University

In recent years there has been a movement away from using contingent valuation to

estimate non-use values towards using various forms of conjoint analysis. Conjoint analysis is

becoming the technique of choice in major government sponsored valuation exercises both in the

US and abroad. This movement in part reflects concerns about possible biases associated with

contingent valuation that are assumed to be less prevalent in conjoint analysis (Hanley, et al

1998).

An important part of the contingent valuation literature was the development of an

incentive compatible provision rule that is made explicit to survey respondents (Arrow, et al,

1993). With conjoint analysis applications, however, respondents are simply asked to reveal

their preferences through various evaluation tasks. Studies will describe a payment mechanism

(such as a tax-price or a user-fee), however the actual rule used to determine which of the

options presented in the survey will be the option that is implemented, if any at all (the provision

rule) is left unspecified.

This study investigates the impacts of provision rules within conjoint choice

questionnaires. Using both private and public goods, we collect conjoint choice data using three

different provision rules and ten different treatments. First, we use an incentive-compatible,

individual provision rule (IPR) involving real payments and purchase. We then conduct a

treatment using an individual provision rule, but with hypothetical payment. Next, we use a

group provision rule (GPR) in which the option that receives the greatest support in the survey is

the option that is actually provided to every subject, regardless if this was his or her preferred

option. This provision rule is not incentive compatible, but important to understand as it mimics

the likely inferred provision rule in past conjoint surveys valuing public goods. Lastly, we

conduct treatments where no provision rule (NPR) is described to subjects. This treatment is

consistent with all previous conjoint applications for environmental valuation. Because no

provision rule is specified, this treatment can not be conducted in an actual-payment scenario --
7

only hypothetical surveys can be conducted.

The treatments we conduct allow us to (1) examine the differences in choices due to

hypothetical versus real-payments (i.e., explore hypothetical bias), an issue that has received

considerable attention in the contingent valuation literature, (2) examine the effects of moving

away from incentive compatibility toward mechanisms that are more realistic (but not incentive

compatible) for conjoint exercises valuing public goods, and (3) examine whether the results

from the treatment with no decision rule (NPR) converges on the results from the incentive

compatible decision rule (IPR) or the group decision rule (GPR).

Preliminary results, based on nearly 2,000 subjects indicate that provision rules have

important effects on the responses to choice surveys. Results indicate that in surveys using

private goods, subjects opt to purchase the private good more often when either the nonincentive compatible group-provision rule is described (GPR), or when no provision rule is

described (NPR). Results from surveys using a public good as the object of choice indicate a

similar pattern. In particular, conditional logit and random parameter logit models indicate that

the provision rule treatment affected the marginal values subjects revealed in the surveys. 

Subjects were significantly less responsive to price in both the GPR and NPR treatments as

compared to the IPR treatment.

Lastly, preliminary results comparing the results from the IPR treatment in which

payments are hypothetical with the IPR treatment in which subjects actually pay for the good

and receive it as a result of their decisions in the choice survey, indicate significant differences in

behavior between these two treatments. Interestingly, while it is clear that subjects “opt out” of

the market more frequently when actual payments could result from their decisions, there may

not be significant differences in the subjects’ responsiveness to prices across the two treatments. 

However, these results are very preliminary in nature at this time.
Internet-Based Stated Choice Experiments in Ecosystem Mitigation:

Methods to Control Decision Heuristics and Biases

John P. Hoehn,

Department of Agricultural Economics

Frank Lupi,

Department of Agricultural Economics and

Department of Fisheries and Wildlife

and

Michael D. Kaplowitz,

Department of Community, 

Agriculture, Recreation and Resource Studies

Michigan State University

East Lansing, MI 48824

Acknowledgments

The reported research was supported, in part, by the: U.S. Environmental Protection Agency,

Science to Achieve Results (STAR) Grant #R827922; the Michigan Sea Grant Program; and the

Michigan Great Lakes Protection Fund. All opinions expressed and errors herein are the authors'

alone.

November 12, 2004

 8 
Abstract

Internet-Based Stated Choice Experiments in Ecosystem Mitigation:

Methods to Control Decision Heuristics and Biases

John P. Hoehn, Frank Lupi, and Michael D. Kaplowitz.

Michigan State University

The research developed internet-based stated choice questionnaires to evaluate wetland ecosystem

mitigation and restoration. The goals were to estimate the in-kind values of ecosystem attributes and

test hypotheses about the performance of the questionnaires. A key question was whether the

ecosystem information and program descriptions were sufficiently detailed to meet the informational

needs of respondents, without overwhelming them with too much information. Behavioral research

shows that respondents’ decisions are inconsistent and biased when confronted with too much

information. The research used a multistage design process to reduce the informational and choice

complexity perceived by respondents. The final questionnaire presented ecosystem information in

a tabular format that enabled respondents to easily identify the choice attributes and to compare

attribute levels and qualities across wetland pairs. A text format was developed as a control to

determine the degree that the tabular format controlled decision heuristics and biases. 

Results indicated that the tabular format was successful in simplifying a complex choice without

eliminating relevant information. In-kind values estimated with the tabular data were consistent

with intuition and statistically significant. In contrast, text format responses were insensitive to high

quality wetlands and highly sensitive to poor quality wetlands, as expected when loss aversion

biases are present. Text responses were also more variable than the tabular responses. The results

suggest that a systematic questionnaire design process reduces the subjective complexity of

ecosystem choices without reducing the objective quality of information. 

 9 
Internet-Based Stated Choice Experiments in Ecosystem Mitigation:

Methods to Control Decision Heuristics and Biases

Wetland ecosystems are regulated under an array of Federal and state regulations. The goal

of many state regulations is similar to the Federal objective of “no net loss” of wetlands (National

Research Council 2001). To avoid a net loss of wetland services, Federal and state regulations may

require mitigation for activities that impair or destroy wetlands. Mitigation raises the issue of

determining what and how much should be done to offset the loss or impairment of a wetland. One

way that losses are offset is by restoring wetlands in locations near a destroyed or impaired wetland.

The amounts and types of restoration are typically determined on ecological grounds. However, a

purely ecological assessment may not adequately address wetland attributes that are valued by

human beings. If the latter values are overlooked, a net economic loss may be incurred despite the

no-net-loss goal. 

Previous research shows that the ecological qualities of wetlands are indeed valued by

ordinary citizens (Heimlich et al. 1998; Kosz 1996; Phillips, Haney, and Adamowicz 1993; Stevens,

Benin, and Larson 1995). Both use and nonuse values are recognized as important to the economic

value of wetlands (Woodward and Wui 2001). Previous research is less clear about the values of

specific wetland attributes and qualities, such as wildlife habitat or access for recreation by the

public. Reported research tends to focus “on the question of ‘what is the value’ and not enough on

what, in particular, people value” (Swallow, 1998, p. 17). 

Identifying the relevant wetland ecosystem attributes is important for both policy and

valuation. Wetlands are complex ecosystems that may be evaluated in different ways. Different

technical approaches characterize wetlands using different metrics and different attributes, such as

hydrogeomorphic types, wetland functions, wetland processes, and ecological values (National

 10 
Research Council 2001). No net loss policies may mistakenly result in real and costly losses when

wetland policies ignore economically important attributes, or use metrics that are only partially

correlated with attributes that are valued by the general public. 

Identifying the subset of economically relevant attributes is also important to the reliability

of stated choice experiments. Recent experiments show that choice complexity reduces the

consistency of stated choices, and increases the variance of stated choice results (Breffle and Rowe

2002; DeShazo and Fermo 2002; Swait and Adamowicz 2001). Including irrelevant attributes and

ill-defined attributes makes stated choices unduly complex. Such complex information sets are

likely to increase the variance of responses and reduce the statistical significance of estimated

values. 

Behavioral research also indicates that complexity increases the respondents’ use of

simplifying decision heuristics (Payne, Bettman, and Schkade 1999). Loss aversion is one of the

common decision heuristics recorded in behavioral research (Kahneman 2003; McFadden 1999).

Faced with a complex decision involving both losses and gains, loss-avoiding respondents make

decisions that myopically avoid losses, and fail to account for gains. Loss-avoiding respondents

tend to focus only on the potential losses and ignore potential gains. Such complexity-induced

heuristics are likely to result in severely biased value estimates to the extent they are evoked by

unnecessarily complex stated choice experiments.

The research reported below developed and tested internet-based questionnaires as a means

of eliciting in-kind values for wetland mitigation from members of the general public. The

questionnaire design drew on behavioral research for strategies to reduce the choice complexity

perceived by respondents. These strategies were incorporated into a four-stage questionnaire design

 11 
process. Focus groups were used to identify the wetland attributes that were most salient to

respondents. Group and individual interviews were used next to test alternative information and

choice formats. Verbal protocol analysis was employed to identify questionnaire content and

attributes that were confusing or misleading to respondents. The final questionnaire presented

ecosystem information in a tabular format that made it easy for respondents to identify choice

attributes and to compare attribute levels across wetland pairs. A conventional text format was

developed as a control to determine the degree that the tabular format controlled decision heuristics

and biases.

Data from the tabular and text formats was used to estimate a mitigation equation that gave

the acreage of a restored wetland necessary to compensate respondents for the loss of an existing

wetland. The amount of restored acreage was conditioned on the acreage of the destroyed wetland,

the quality differences between the two wetlands, and the demographic characteristics of

respondents. With the data from the text format, the estimated in-kind values were consistent with

uncontrolled loss aversion bias. Text format responses were highly sensitive to poor quality wetlands

and insensitive to both wetland size and high quality wetlands. In contrast, tabular responses were

sensitive to wetland size, low quality wetlands, and high quality wetlands. The tabular format

appeared to facilitate informed and balanced tradeoffs. 

Economic Model of Mitigation Choices

Wetland mitigation compensates for the loss of wetland services with the restoration of

wetland services in a different location. As such, wetland mitigation offers a natural setting for

eliciting pair-wise stated choices between a restored and drained wetland. An individual may be

asked whether a restored wetland of a given acreage and quality is sufficient to compensate for the

 12 
(1)

(2)

loss of a destroyed wetland of a given acreage and quality. This section presents a utility theoretic

framework for such choices and derives the in-kind values associated with wetland acreage and

wetland ecosystem qualities.

Stated preference techniques are widely applied in market research (Louviere 1991),

transportation economics (Bates 2000), development economics (Rubey and Lupi 1997), and

environmental economics (Adamowicz et al. 1998; Boxall et al. 1996; Mackenzie 1993; Opaluch

et al. 1993; Swallow et al. 1998). Stated choices are usually estimated within a random utility

formulation. In this analysis, we derive a choice model based on offered restoration versus a desired

amount of utility compensating restoration. 

The analysis begins with the preferences of a respondent drawn from the general public. The

respondent has preferences over wetland size and wetland qualities that are conditioned on the

respondent’s demographic characteristics. These preferences are summarized by a utility function,

defined on wetland acreage, , a K-element vector denoting the quality of wetland services, , and

a N-element vector of individual respondent characteristics, c. 

Consider the loss of a wetland that is acres in size with qualities . The amount of

restored acreage, m, with qualities , that compensates for the loss of with qualities is

 13
(3)

(4)

(5)

Equation (2) states the amount of compensatory restoration, m, as an implicit function of lost

wetland acreage, lost and restored wetland service qualities, and individual respondent

characteristics. 

A compensatory mitigation equation is derived by inverting the left-hand side of equation

(2) about the amount of restored acreage, 

Equation (3) may be rewritten as a mitigation function,

Equation (4) is similar to an income compensation function (Chipman and Moore 1980) except that

the mitigation compensation function is denominated in restored acreage rather than income. The

mitigation equation states the amount of quality adjusted restored acreage required to compensate

for the loss of an existing wetland of a given size and quality.

The mitigation function is approximated with a linear function of destroyed wetland acreage,

the difference between the qualities of the restored and destroyed wetland, respondent demographic

characteristics, and a stochastic term,

where is an intercept coefficient, is the coefficient of the acreage of the destroyed wetland,

; is the coefficient of the difference between the restored and destroyed wetland in the kth

 14
(6)

wetland quality, ; is the coefficient of the nth respondent characteristic, , such as income

level or having never visited a wetland; and is a stochastic error term. The stochastic term, ,

represents random choice effects that are unobserved by the researcher.

The stated choice experiments present respondents with a pair of wetlands, the restored

wetland with its quality attributes, and the destroyed wetland with its quality attributes. The

respondent can either accept or reject the restored wetland as compensation for the loss. A

respondent accepts the restored wetland as compensation if the size of the restored wetland is greater

than the compensating mitigation described by equation (5), given the size of the destroyed wetland,

the quality differences between the restored and destroyed wetland, and individual characteristics.

A respondent rejects the restored wetland as compensation if the size of the restored wetland is less

than the compensatory mitigation described by equation (5). 

Given the stochastic term in equation (5), a respondent’s decision is not known with certainty

by a researcher. However, the probability that an individual accepts restored acreage with

qualities is

When the stochastic term, , is an independently distributed normal random variable, the probability

of accepting the offered restored wetland is

 15
(7)

(8)

where is the standard normal cumulative density and is the standard deviation of the

stochastic term . Equation (7) describes a model similar to ordinary probit. However, in the

ordinary probit model, the standard deviation is not identified and the variable coefficients are

identified only to a scale factor. In equation (7), the coefficient of restored acreage is one, so the

coefficient of restored acreage estimated by an ordinary probit is (Cameron and James 1987).

Thus, the form of the mitigation equation identifies and the other coefficients of the mitigation

equation. The mitigation coefficients may be estimated as simple ratios of the probit coefficients

and standard errors may be computed using a Wald procedure (Greene 2000).

In stated choice, it is convenient to elicit multiple choices from the same respondent. In this

case, responses may not be independent due to the possibility that a respondent’s choices may vary

in a systematic manner. Butler and Moffitt (1982) show that equation (7) may be rewritten

conditionally on a random respondent effect u,

where is the standard deviation of the random individual effect, is the standard unit normal

random variable, , and is the standard deviation of the cross-section stochastic term,

, representing unobserved and independently distributed choice effects. 

 16
Equation (8) is a density function conditioned on the random variable representing the

individual effect. The random effects probit model is derived by setting up the likelihood equation

for the ordinary probit and computing the expectation of the likelihood equation with respect to .

The expected likelihood equation is then evaluated by Gaussian quadrature to obtain maximum

likelihood estimates of the coefficients and standard deviations (Butler and Moffitt 1982). 

Stated Choice Questionnaire

The purpose of the questionnaire design process was to develop questionnaires that

accurately describe wetland qualities and choices to respondents, while controlling the perceived

complexity of wetland information and tradeoffs. Previous research shows that such complexity can

introduce inconsistencies across stated choices (Breffle and Rowe 2002; DeShazo and Fermo 2002;

Swait and Adamowicz 2001) and potentially lead to characteristic biases due to loss aversion and

other decision heuristics (Kahneman 2003; McFadden 1999). 

Stated choice research tends to treat complexity as an objective phenomenon (Breffle and

Rowe 2002; DeShazo and Fermo 2002; Swait and Adamowicz 2001), but behavioral research

indicates that it is subjective and conditioned on the structure of a particular informational treatment

(Carlson, Chandler, and Sweller 2003; Ganier, Gombert, and Fayol 2000; Simon 1974). An

informational treatment may focus on relevant information or it may force a reader to sort through

relevant and irrelevant data. Focusing on the important and salient features reduces the number of

features to be evaluated by respondents and thereby reduces one aspect of complexity. In addition,

the same information may be presented in complex or simple ways. For example, the structure of

human memory appears to make it easier for people to remember more information when it is

 17
presented with both text and graphical representations than when it is presented solely as text (Luck

and Vogel 1997). 

The research used the four-stage process described in Figure 1 to build the lessons of

behavioral and cognitive research into a stated choice questionnaire. The objective of the first stage

of the process was to determine the types of information that respondents find most relevant to

mitigation choices. The first stage examined the general public’s knowledge about wetlands, their

experience with wetlands, and the priorities they placed on protecting and managing specific

wetland qualities and services. 

Data for the first stage analysis was obtained in focus group discussions. Participants were

recruited using telephone numbers drawn randomly from the Lansing-area phone book.

Respondents were asked whether they could attend a discussion group concerning ‘critical policy

issues’. Respondents were selected so that the mix of focus group participants was representative

of the demographic characteristics of mid-Michigan. Focus group discussions were guided by

trained moderators using a written discussion guide. The discussion guide began by eliciting top

environmental concerns, and then probing these concerns to assess their connections with aquatic

and wetland ecosystems. The discussion guide gradually probed the topic of wetlands and elicited

respondents’ baseline knowledge and experience with wetlands. The last segment of the discussion

guide presented a wetland mitigation case and asked respondents to evaluate the case in terms of the

adequacy of compensatory mitigation.

Results from the first stage group discussions were used to construct alternative information

and choice formats. The alternative formats varied in three dimensions. The first dimension was

the way the formats used hierarchical taxonomies to categorize wetland attributes. Behavioral

 18
research shows that information is readily assimilated and used when it is organized in subjectively

meaningful categories or chunks (Gobet et al. 2001; Simon 1974). For instance, it is more difficult

to recall the letters BMICSIACB than when they are arranged in as the chunks IBM-CIA-CBS

(Bettman, Payne, and Staelin 1987). Similarly, wetland species may be chunked into taxonomic

categories, such as wading birds, song birds, and amphibians. A format that divides species into

species categories and lists specific species within that category is likely to appear less complicated

to respondents than a format that simply lists individual species. Such a hierarchical listing may

also appeal jointly to non-expert respondents and lay experts. Non-expert respondents may focus

on the species categories while lay experts may find the sets of specific species meaningful to their

evaluations and choices.

The second dimension that varied across the alternative formats was the way that the restored

and destroyed wetlands were presented. For instance, the information about each of the two

wetlands may be presented sequentially on separate pages or on the same page. Tabular designs

may array wetland attribute information in corresponding columns to facilitate comparisons across

the destroyed and restored wetlands. Previous research indicates that tabular designs reduce the

perceived task complexity and reduce the amount of time needed for task completion (Carlson,

Chandler, and Sweller 2003; Ganier, Gombert, and Fayol 2000). 

The third dimension was a method of describing quality changes across wetlands. Focus

group respondents tended to describe wetland experiences in terms of what they saw as they drove

by or walked through wetlands. These comments suggested that metrics based on transect sampling

may be a meaningful way to represent quality differences between wetlands. Transect sampling

plots a path through a given area and counts all the features of interest along that path within a given

 19
time (Buckland et al. 1993). A wetland visitor might be thought of as conducting an informal

transect where the quality of the wetland ecosystem influences the chance of seeing different

categories of species, so one of the quality indicators evaluated was based on the chance that a lay

visitor might see certain species during a visit to the wetland. 

The second stage of the questionnaire design process evaluated the alternative information

treatments in group and individual interviews. Participants for the group and individual sessions

were drawn from the Lansing area through random selection using a local telephone directory. The

interviews were divided into interview and debriefing segments. During the interview segment,

respondents completed one of the alternative questionnaires. The debriefing segment began once

the questionnaires were completed. The debriefing segment was led by a trained moderator who

followed a written discussion guide. The guide asked respondents to discuss how they understood

the tasks required by different parts of a questionnaire, examined any difficulties that respondents

had in completing a questionnaire, and ended with a short quiz to assess respondents’

comprehension of the questionnaire. 

The debriefing interviews identified one questionnaire design as superior to the others. The

questionnaire used hierarchical categories, a tabular layout, and placed the qualities of the drained

and restored wetlands in adjacent columns on a single page of the questionnaire. Respondents using

this format made specific and repeated references to wetland qualities during the debriefings. No

specific quality seemed to dominate their recollections. Rather, respondents seemed to have

balanced and nuanced perceptions of the wetland attributes being compared. The tabular format

appeared to facilitate choice-related comparisons and tradeoffs across the two wetlands. Several

 20
respondents commented that the choices were almost “too easy,” despite the fact that the choices

involved nine attributes, three possible quality levels for each attribute, and two wetlands. 

The third stage of the questionnaire design process used individual pretest interviews to test

and revise the prototype questionnaire. Pretest respondents were again drawn from the Lansing, MI

area using a random selection method based on area telephone books. Participants were paid an

honorarium to attend the pretest at a specified day and time on the Michigan State University

campus. Pretest interviews were divided into questionnaire self-administration and debriefing

segments. In the survey self-administration segment, each participant completed a prototype

questionnaire. Once the questionnaire was complete, the respondent was guided to a private office

for an in-depth debriefing interview. 

Debriefing interviews followed a detailed written debriefing guide. The guide began by

asking a respondent to recall and describe their thoughts as he or she completed particular segments

of the questionnaire. Additional questions focused on how and whether the respondent understood

the information and choice segments of a questionnaire. The debriefing ended with several

knowledge-based questions to determine whether respondents understood important aspects of the

questionnaire and choice question. Debriefing data were used to revise the prototype questionnaire,

with the resulting revised questionnaires subjected to further testing. 

The final stage of the design process programmed the questionnaire for use on the internet.

The programming was done to preserve the appearance of a paper questionnaire as much as possible.

The draft internet questionnaire was pretested over the internet with respondents from the Lansing

area, primarily to test the technical characteristics of the questionnaire. Respondents were recruited

through random identification from telephone records and paid an honorarium to complete the

 21
questionnaire and debriefing interview. Respondents used a variety of operating systems, e-mail

systems, web browsers, computer displays, and different internet service providers. Despite the

wide range of situations, relatively few problems arose with the web-based questionnaire. The minor

problems that did arise were readily remedied by some minor reprogramming.

The final questionnaire focused on a subset of wetland attributes and presented these

attributes in a tabular design. Attributes were selected for inclusion based on the data obtained in

the qualitative research and pretesting. Attributes included wetland size, type of vegetative cover,

accessibility by the general public, and suitability as habitats for plant and animal species.

Vegetative cover was categorized as marsh, wooded wetland, or a mix of both marsh and wooded

wetland.

Figure 2 shows a portion of the final tabular information format included in the final

questionnaire. The tabular form arrayed the relevant wetland choice information in two adjacent

columns, one for each wetland under consideration. Wetland habitats were described in five

dimensions; habitat quality for amphibians and reptiles, habitat quality for small mammals, habitat

quality for song birds, habitat quality for wading birds, and habitat quality for wild flowers. 

Each type of habitat was described with a rating of poor, good, or excellent based on the

transect sampling concept discussed above. Habitat quality ratings were provided for both the

drained and restored wetlands. A narrative box at the bottom on the table explained each of the

quality ratings. The ratings were based on what a visitor was likely to see during a visit to the

wetland. A poor rating was indicated by “--” and was defined as a wetland habitat that supported

“these species in very small numbers...[so] a trained observer is unlikely to find any of these

species.” A “good” rating meant that the wetland habitat supported “these species in average

 22
numbers...[so] a casual observer is likely to see a few of these species.” An “excellent” rating meant

that the wetland habitat supported “these species in better than average numbers...[so] a casual

observer is very likely to see a variety of these species.”

Internet-Based Stated Choice Experiments 

The objective of the internet experiment was to test the performance of the developed questionnaire.

The hypothesis was that the tabular format reduced complexity and encouraged reasoned decisions

informed by a balanced view of all wetland attributes. To test the hypothesis, an experimental

control was developed based on a text version of the information format. The text version contained

information that was objectively identical to the tabular questionnaire. The only difference between

the tabular and text questionnaires was that the text format used sentences to convey the information

about wetland attributes and qualities. Figure 3 gives an example of the text format.

Two empirical consequences were expected if the tabular format reduced the complexity

perceived by respondents. First, prior research showed that reduced complexity increases the choice

consistency and reduces the variance of choice responses (Breffle and Rowe 2002; DeShazo and

Fermo 2002; Swait and Adamowicz 2001). In the present case, reducing perceived complexity

should result in greater consistency and smaller standard deviations for both the cross-section effect,

, and the respondent effect, . Thus, the estimated standard deviations for the tabular format data, 

and , should be smaller than the estimated standard deviations for the text format data.

 Second, behavioral research indicates that complexity leads to increases in the use of

decision heuristics and biases (Payne, Bettman, and Schkade 1999). Viscusi and Magat (1987)

found that text formats had less impact on risk avoidance behavior and willingness to pay than

tabular formats. Psychological research stresses that cognitive constraints lead to characteristic

 23
biases when dealing with complicated decisions (Kahneman 2003; Payne, Bettman, and Schkade

1999). Loss aversion is one characteristic and common decision bias (Kahneman, Knetsch, and

Thaler 1991; McFadden 1999). With loss aversion, respondents overweight losses and underweight

gains. 

With complex wetland choices, it was hypothesized that loss aversion would lead

respondents to overweight wetlands with poor quality attributes and underweight wetlands with

excellent quality attributes. As a result, a mitigation equation estimated with the text data was

expected to have larger coefficients for variables indicating poor quality than a mitigation equation

estimated with the tabular data. In contrast, a mitigation equation estimated with text data would

be expected to have smaller coefficients for variables indicating excellent quality. The text

coefficients for variables indicating excellent quality may be ignored by respondents, with the result

that their coefficients may not be statistically different from zero. In contrast, the statistical

significance of the coefficients estimated with tabular data is likely to be more evenly distributed

across poor and excellent quality indicators. 

Data to estimate the mitigation equations and test these empirical hypotheses was collected

in a large-scale internet experiment with Michigan residents. Access to a panel of potential webbased respondents was purchased from Survey Sampling International (SSI), a commercial provider

of sampling frames and databases. The SSI panel is a self-selected sample of potential respondents

with known demographic characteristics.

The web-based experiment was implemented in multiple stages beginning in October and

ending in December, 2003. E-mail invitations to 16,936 members of the SSI panel, resulted in 3,420

clicks on a welcome page to the web-based questionnaire. From the welcome page, 25 percent of

 24
respondents were randomly assigned to the text format. In all, 2,689 respondents began the first

page of the questionnaire. Usable questionnaires with at least one completed mitigation choice

and complete demographic information numbered 1,326. This was 8 percent of the number of email invitations and 40 percent of those visiting the welcome page. Eight percent is a midrange rate

for recent internet experiments (Berrens et al. 2002). 

Results

The tabular and text formats yielded two sets of data suitable for an analysis of mitigation choices

and values. The data pertaining to the tabular format were the preferred, core data set, since the

tabular design was subject to the full iterative design process. The purpose of the text format data

was to provide a baseline for evaluating the performance of the tabular design. By hypothesis, the

text format leads to (1) more inconsistency in stated choices and (2) cognitive biases that overweight

losses in wetland qualities and underweight gains in wetland quality.

The text and tabular data contained three types of variables. First, there were the wetland

choice variables. Respondents were given five mitigation scenarios and were asked to determine

whether the restored wetland was sufficient to offset the loss of a drained wetland. Hence, each

individual recorded accept or reject choices for up to five restoration scenarios. Second, there were

the variables that described the acreage and qualities of both the drained and restored wetlands.

Third, there were demographic variables for each respondent. 

Table 1 lists demographic characteristics for respondents to the tabular and text versions of

the questionnaire. There were 937 respondents to the tabular version and 363 respondents to the text

version who had responses complete enough to the used in the choice analysis. The choice analysis

required complete responses for the variables listed in Table 1.

 25
1

The sample selection procedures were intended to be weighted by the Census proportions

for males and females in the 2000 Census. However, an error occurred in subcontractor’s sample

selection process during the waves 1 and 2 of the experiment. The error was corrected for waves

3 to 6 and the sample size was increased to meet the demographic criteria for the initial sample

design. 

Mean levels of income, education, age, and gender were similar for respondents to both the

tabular and text versions. One exception was for the age of respondents where the text data set

contained about 8 percent more respondents who were over 65 years of age. The mean income level

for respondents to both versions was about the same as the 2002 Census mean for the State of

Michigan. Respondents to the questionnaires were somewhat more schooled with some college

study and were more likely to be female and over 65.1 Finally, 15 percent of the respondents in each

sample had never visited a wetland.

The tabular and text data was used to estimate a mitigation equation (5) using the random

effects probability model of equation (8). Table 2 lists the general characteristics of the two

estimated equations. The data included 4,685 choices from the 963 respondents who used the

tabular format and 1,811 choices from the 363 respondents who used the text format. The tabular

and text equations performed about equally well in predicting both yes and no responses.

The tabular and text equations are noticeably different in the standard deviations for both the

cross-sectional and respondent effects. The standard deviations for the text data are more than twice

the size of those for the tabular data. The third column shows that the differences between the two

sets of standard deviations are statistically different from zero at the 90 percent level of significant.

These results indicate that respondents make more consistent choices with the tabular questionnaire

format than the text questionnaire format. The tabular format appears to be successful in reducing

perceived complexity, at least as indicated by the variability of choices.

 26
Table 3 lists the wetland attributes and demographic variables used to estimate the mitigation

equation coefficients. Wetland size was one of the variables and ranged from 5 to 19 acres for the

drained wetlands and from 4 to 48 acres for the restored wetlands. Other wetland characteristics

were described as categorical variables. The drained and restored wetlands (a) allowed access by

the public, denoted by a “yes,” (b) allowed access to the pubic with developed trails, denoted by

“yes-trails,” or (c) made no provision for public access, denoted by “no.” The type of wetland was

either a marsh, a wooded wetland, or a mixture of marsh and woodlands.

The changes in wetland characteristics variables, , were transformations of the data in

the questionnaires. The change in access variable indicated whether there was a change in public

access in the restored wetland relative to the drained wetland. The change in access variable was

given a value of 1 if the restored wetland allowed public access while the drained wetland did not.

Change in access was -1 if the restored wetland did not provide for public access while the drained

wetland did provide for public access. In other cases, change in access was set to 0. 

The change in wetland type variable was a simple, unsigned dummy variable. It was given

a value of 1 if there was a change in wetland type between the restored and drained wetlands and

set to 0 if there was no change in type.

The changes in wetland habitat variables were computed from dummy variables representing

the poor and excellent categories. The first step was to assign a dummy variable for each of the poor

and excellent quality levels of the drained and restored wetlands. Each of the “poor” dummy

variables was given a value of 1 if a particular habitat category was poor in quality, and was set to

zero otherwise. Each of the “excellent” dummy variables was given a value of 1 if a particular

habitat quality was excellent in quality, and was set to zero otherwise. Dummy variables were

 27
2

The small animals habitat quality was kept constant across the two wetlands to reduce the

size of the experimental design. Because the small animals are generalists, this type of habitat was

not thought to vary across substantially across the common wetlands under consideration, and the

other habitat categories were sufficient to demonstrate the role of habitat quality with respect to

respondents’ preferences.

created for the “poor” and “good” variables for four habitats (reptiles/amphibians, song birds,

wading birds, and wild flowers) and both wetlands, so there were 8 initial dummy variables for

quality. The habitat dimension for small animals was kept constant across the choice experiments,

so no dummy was created to indicate the quality of habitat for small animals.2

The second step in computing the habitat change variables was to compute the difference in

the habitat dummy variables between the restored and drained wetlands. For instance, the change

in poor dummies for reptiles/amphibians was the difference between (a) the poor reptiles/amphibians

dummy for the restored wetland and (b) the poor reptiles/amphibians dummy for the drained

wetland. A value of 1 for the latter variable meant that the reptiles/amphibian habitat was poor for

the restored wetland and not poor for the drained wetland. A value of -1 meant that the

reptiles/amphibian habitat was not poor in the restored wetland and poor in the drained wetland.

A value of 0 meant no change in the habitat quality for the reptiles/amphibians habitat across the two

wetlands. Similar habitat change variables were computed for computed for the poor and excellent

dummies variables for each of the 4 habitat categories, resulting in 4 variables to reflect changes in

poor quality habitat and 4 variables to reflect changes in excellent quality habitat. 

The demographic characteristics variables were simple levels or categorical dummy

variables. Income was measured in thousands of dollars. The remainder of the respondent variables

were categorical dummy variables, taking the value of 1 if the respondent had the characteristic, and

taking the value of 0 otherwise. 

 28
Table 3 displays the estimated mitigation coefficients for the tabular and text equations. The

second and third columns list the estimated normalized coefficients for the tabular and text data.

The final column lists the differences between the coefficients of the tabular and text coefficients.

The coefficients for the tabular equation have plausible signs and are mostly statistically different

from zero at the 95 percent level. The normalized coefficient for drained acreage is equal to 1.42.

A acreage coefficient equal 1 would mean that restored wetland acreage is a very close substitute

for drained acreage. However, the coefficient is 42 percent larger than one and statistically different

from 1 at the 95 percent level. The coefficient implies that the mean respondent requires

compensation of 1.42 restored acres for each acre of drained wetland, even when the two wetlands

are otherwise identical in access, wetland type, and habitat quality. 

The premium of 42 percent on the drained wetland acreage is similar to Mullarkey’s finding

that natural wetlands are more valuable than restored wetlands (Mullarkey 1997). However,

Mullarkey found a much larger premium on dollar value of natural wetlands, perhaps due to

unaddressed differences in wetland qualities.

Public access and wetland type also have a significant impact on the amount of mitigation

acreage that compensates for loss of the drained wetland. The public access coefficient indicates

that providing public access reduces the compensating number of mitigated acres by 5.76 acres. A

change in wetland type increases the compensating amount of mitigation by 4.69 acres.

The change in habitat variables are all significantly different from zero for the tabular data

and have algebraic signs consistent with intuition. Reductions in habitat qualities from good to poor

require additional acreage to offset the loss in quality. A change in a reptile/amphibian habitat from

good to poor requires 8.19 additional restored acres to offset the loss of quality. A reduction in a

 29
wild flower habitat from good to poor requires 2.33 acres of additional restored acreage. 

Improvements in habitat quality relative to the drained wetland reduce the amount of restored

acreage required for mitigation. A change from a good wading bird habitat in the drained wetland

to an excellent habitat in the restored wetland reduces the number of restored acres by 5.09 acres.

An improvement from a poor habitat in the drained wetland to an excellent habitat in the restored

wetland is assessed by summing the appropriate coefficients. For instance, for song birds, a change

from poor to excellent reduces the number of restored acres by 6.56 plus 3.80, an overall reduction

of 10.36 acres.

Several demographic characteristics affect the level of mitigation that compensates for

wetland loss. Increases in respondents’ income and schooling tend to reduce the size of

compensatory mitigation projects. Having visited a wetland at some point in the past also leads to

reductions in the amount of compensating mitigation acres. The latter variable is interesting since

it indicates that individuals who have some experience with common wetlands are more inclined to

accept the replacement of existing wetlands with restored wetlands. 

The notable feature of the text coefficients is the large size of the poor quality habitat

coefficients and the small size of the excellent quality habitat indicators. Respondents who were

randomly given the text-based choice question require more acreage compensation for loss in

quality than the respondents who were randomly selected to receive the tabular-based choice

question. Alternatively, for improvements in restored habitat quality relative to the drained wetland,

text respondents behave in just the opposite fashion; they underweight improvements. 

The final column of Table 3 shows that these asymmetries are statistically significant for

each of the poor habitat coefficients and are significant as a group for the excellent habitat

 30
coefficients. The results suggest that relative to the tabular format the text respondents fell prey to

decisions biases that have been noted by psychologists: respondents tend to overweight losses and

underweight gains. The tabular questionnaire appears unaffected by such biases. Coefficient

estimates are relatively precise and the differences between coefficients seem reasonable and

consistent with intuition. The iterative design process appears successful in deriving a questionnaire

that supported balanced, reasoned decisions for rather complex mitigation choices.

The strong asymmetry in the resulting data from the text choice questionnaire also appears

in estimating mitigation acreage requirements. Suppose one is considering mitigation for the

drainage of a 20-acre wetland with good habitat quality in each of the four habitat categories.

Consider two restoration projects: the first involves restoration that results in all four habitat

qualities being in poor condition, and the second involves restoration that results in all four habitat

qualities being in excellent condition. In the first case, the mitigation equation estimated with the

tabular data requires 49 acres of restored wetland acres as compensation, but the equation estimated

with the text data requires 106 acres of restored wetland as compensation. Conversely, in the second

case involving restoration with excellent habitat quality, computing compensating restoration

acreage with the tabular equation requires 11 acres of compensation while the text equation requires

28 acres as compensation. 

The mitigation examples highlight the differences between the text and tabular data, and the

hypothesized superiority of the tabular questionnaire. With the text questionnaire, respondents

appear to overweight losses in habitat quality and underweight gains. The underweighting of gains

is rather extreme, since the individual habitat coefficients for improvements are small in size and

statistically indistinguishable from zero. In contrast, the tabular data results in coefficients that are

 31
economically significant, statistically different from zero, much more balanced in their assessment

of wetland gains and losses, and accord with the respondent feedback from the focus groups and

pretest interviews.

Conclusion

The research demonstrates that stated choice experiments with complex ecosystems are

feasible for the general public. Careful research on baseline knowledge and systematic pretesting

appear essential for obtaining reasonable, unbiased stated choice results. The tabular questionnaire

format that resulted from a four-stage design procedure appeared to perform well. The research also

used a simple text-based information treatment as an example of the type of questionnaire that might

be developed without the iterative questionnaire design process. The simple text-based

questionnaire revealed the kinds of asymmetric biases anticipated on the basis of recent

psychological and economic research (McFadden 2001). The text-based descriptions resulted in

losses in ecosystem quality being overweighted and gains in quality being underweighted relative

to those estimated using the tabular format. Thus, while ecosystem choices may be complex enough

to strain respondents= decision capacities, systematic questionnaire development seems able to help

researchers arrive at formats that reduce or eliminate the impact of characteristic biases on the estimated

values.

The results demonstrate that wetland qualities and services are valued by members of the general

public. From qualitative research, wetland habitats for small animals, birds, and special plants were

found to be of special interest and value to respondents (Hoehn, Lupi, and Kaplowitz 2003; Kaplowitz,

Lupi, and Hoehn 2004). Respondents had direct experience with the latter types of wetland habitats and

saw them as directly impacted by mitigation activities. The importance of habitat quality emerged

 32
consistently at all stages of the research including the initial focus groups, the pretest phase, and the

web-experiments. This finding is similar to other recent research on wetland ecosystems (Azevedo,

Herriges, and King 2000; Johnston et al. 2002; Stevens, Benin, and Larson 1995; Swallow et al. 1998).

Two aspects of the research need to be kept in mind in interpreting the results. First,

respondents to both the qualitative and quantitative research were drawn from residents of Michigan.

Michigan=s climate is characteristic of the humid north-central portion of the United States. Wetlands

are a common landscape feature, so Michigan residents may have more experience with wetlands than

those in other parts of the United States, especially those living in arid regions. Second, while the study

provides estimates of how to adjust mitigation ratios to account for differences in habitat quality, it

should be considered a first step. The objective of this research was not to estimate values representing

a particular population, but to develop and evaluate stated choice valuation methods and procedures.

Further research is needed to implement the developed procedures in a statistically representative

sample. Third, the wetlands considered here were common types which are regularly subject to permit

actions in Michigan. The study results do not apply to rare wetlands, rare habitats, or rare species.

Likewise, in the wetland choices studied here, respondents were explicitly asked to hold other functions

of wetlands constant. 

 33
References

Adamowicz, W., P. Boxall, J. Louviere, J. Swait, and M. Williams. 1998. Stated Preference Methods

for Valuing Environmental Amenities. In Valuing Environmental Preferences: Theory and

Practice of the Contingent Valuation Method in the U.S., E.C., and Developing Countries,

edited by I. Batemen and K. Willis. Oxford: Oxford University Press.

Azevedo, Christopher, Joseph A. Herriges, and Catherine L. King. 2000. Iowa Wetlands: Perceptions

and Values. Ames: Center for Agricultural and Rural Development Iowa State University.

Bates, John. 2000. History of Demand Modelling. In Handbook of Transport Modelling, edited by D.

A. Hensher and K. J. Button. Handbooks in Transport, vol. 1. Amsterdam; New York and

Oxford: Elsevier Science Pergamon.

Berrens, R.P., A.K. Bohara, H.C. Jenkins-Smith, C.L. Silva, and D.L. Weimer. 2002. Information and

Effort in Contingent Valuation Surveys: Application to Global Climate Change Using National

Internet Samples. Journal of Environmental Economics and Management 47:331-363.

Bettman, J.R., J.W. Payne, and R. Staelin. 1987. Cognitive Considerations in Presenting Risk

Information. In Learning About Risk, edited by K. W. Viscusi and W. A. Magat. Cambridge,

MA: Harvard University Press.

Boxall, P., W. Adamowicz, J. Swait, M. Williams, and J. Louviere. 1996. A Comparison of Stated

Preference Methods for Environmental Valuation. Ecological Economics 18:243-253.

Breffle, W.S., and R.D. Rowe. 2002. Comparing Choice Question Formats for Evaluating Natural

Resource Tradeoffs. Land Economics 78 (2):298-314.

Buckland, S.T., D.R. Anderson, K.P. Burnham, and J.L Laake. 1993. Distance Sampling: Estimating

Abundance of Biological Populations. London: Chapman and Hall.

Butler, J.S., and Robert Moffitt. 1982. A Computationally Efficient Quadrature Procedure for the OneFactor Multinomial Probit Model. Econometrica 50 (3):761-764.

Cameron, T.A., and M.D. James. 1987. Efficient Estimation Methods for 'Closed-Ended' Contingent

Valuation Surveys. Review of Economics and Statistics 69:269-276.

Carlson, R., P. Chandler, and J. Sweller. 2003. Learning and Understanding Science Instructional

Material. Journal of Educational Psychology 95 (3):629-640.

Chipman, J.S., and J.C. Moore. 1980. Compensating Variation, Consumer's Surplus, and Welfare.

American Economic Review 70 (5):933-949.

DeShazo, J.R., and G. Fermo. 2002. Designing Choice Sets for Stated Preference Methods: The Effects

of Complexity on Choice Consistency. Journal of Environmental Economics and Management

44:123-143.

Ganier, F., J.E. Gombert, and M. Fayol. 2000. The Effects of Instructional Formats on Procedural

Learning when Using Technical Documents. Travail Humain 63 (2):121-152.

Gobet, F., P.C.R. Lane, S. Croker, P.C-H. Cheng, G. Jones, I. Oliver, and J.M. Pine. 2001. Chunking

Mechanisms in Human Learning. Trends in Cognitive Sciences 5 (6):236-243.

Greene, W.H. 2000. Econometric Analysis. 4th ed. Upper Saddle River, NJ: Prentice-Hall.

Heimlich, Ralph E., Keith D. Wiebe, Roger Claassen, Dwight Gadsby, and Robert M. House. 1998.

Wetlands and Agriculture: Private Interests and Public Benefits. Washington, D.C.: Resource

Economics Division, Economic Research Service, U.S. Department of Agriculture.

 34
Hoehn, J.P., F. Lupi, and M.D. Kaplowitz. 2003. Untying a Lancastrian Bundle: Valuing Ecosystems

and Ecosystem Services for Wetland Mitigation. Journal of Environmental Management

68:263-272.

Johnston, Robert J., Gisele Magnusson, Marisa J. Mazzotta, and James J. Opaluch. 2002. The

Economics of Wetland Ecosystem Restoration and Mitigation: Combining Economic and

Ecological Indicators to Prioritize Salt Marsh Restoration Actions. Am J Agricultural

Economics 84 (5):1362-1370.

Kahneman, D., J.L. Knetsch, and R.H. Thaler. 1991. Anomalies the Endowment Effect, Loss Aversion,

and Status Quo Bias. Journal of Economic Perspectives 5 (1):193-206.

Kahneman, Daniel. 2003. Maps of Bounded Rationality: Psychology for Behavioral Economics.

American Economic Review 93 (5):1449-1475.

Kaplowitz, Michael D., Frank Lupi, and John P. Hoehn. 2004. Multiple-Methods for Developing and

Evaluating a Stated Preference Survey to Value Wetland Ecosystems. In Methods for Testing

and Evaluating Survey Questionnaires, edited by S. Presser, J. M. Rothgeb, M. P. Couper, J.

T. Lessler, E. Martin, J. Martin and E. Singer. New York: John Wiley and Sons.

Kosz, Michael. 1996. Valuing Riverside Wetlands: The Case of the "Donau-Allen" National Park.

Ecological Economics 16:109-127.

Louviere, J. J. 1991. Experimental Choice Analysis: Introduction and Overview. Journal of Business

Research 23:291-297.

Luck, S.J., and E.K. Vogel. 1997. The Capacity of Visual Working Memory for Features and

Conjunctions. Nature 390 (6657):279-281.

Mackenzie, J. 1993. A Comparison of Contingent Preference Models. American Journal of Agricultural

Economics 75:593-603.

McFadden, Daniel. 1999. Rationality for Economists? Journal of Risk and Uncertainty 19 (1-3):73-105.

McFadden, Daniel. 2001. Economic Choices. American Economic Review 91 (3):351-378.

Mullarkey, Daniel J. 1997. Contingent Valuation of Wetlands: Testing Sensitivity to Scope.

Dissertation, Agricultural and Applied Economics, University of Wisconsin, Madison, WI.

National Research Council. 2001. Compensating for Wetland Losses under the Clean Water Act.

Washington, DC: National Academy Press.

Opaluch, J. J., S. Swallow, T. Weaver, C. Wessels, and D. Wichlens. 1993. Evaluating Impacts from

Noxious Waste Facilities: Including Public Preferences in Current Siting Mechanisms. Journal

of Environmental Economics and Management 24:41-59.

Payne, J.W., J.R. Bettman, and D.A. Schkade. 1999. Measuring Constructed Preferences: Towards a

Building Code. Journal of Risk and Uncertainty 19 (1-3):243-70.

Phillips, W. E., T. J. Haney, and W. L. Adamowicz. 1993. An Economic Analysis of Wildlife Habitat

Preservation in Alberta. Canadian Journal of Agricultural Economics 41 (4):411-418.

Rubey, L, and F. Lupi. 1997. Predicting the Effects of Market Reform in Zimbabwe: A Stated

Preference Approach. American Journal of Agricultural Economics 78:89-99.

Simon, H.A. 1974. How Big Is a Chunk? Science 183:482-488.

Stevens, Thomas H., Samuel Benin, and Joseph S. Larson. 1995. Public Attitudes and Economic Values

for Wetland Preservation in New England. Wetlands 15:226-231.

Swait, J., and W. Adamowicz. 2001. Choice Environment, Market Complexity, and Consumer

Behavior: A Theoretical and Empirical Approach for Incorporating Decision Complexity into

 35
Models of Consumer Choice. Organizational Behavior and Human Decision Processes 86

(2):141-167.

Swallow, S., M. Spencer, C. Miller, P. Paton, R. Deegen, L. Whinstanley, and J. Shogren. 1998.

Methods and Applications for Ecosystem Valuation: A Collage. In Proceedings of the first

Workshop in the Environmental Policy and Economics Workshop Series, edited by B.

Kanninen. Washington, D.C.: Office of Research and Development and Office of Policy, U.S.

Environmental Protection Agency.

Viscusi, W. Kip, and Wesley A. Magat. 1987. Learning about Risk. Cambridge, MA: Harvard

University Press.

Woodward, R.T., and Y-S. Wui. 2001. The Economic Value of Wetland Services: A Meta-Analysis.

Ecological Economics 37:157-270.

 36
Figure 1. Four Stage Questionnaire Design Process 

Stage Methods Result 

 

Stage 1: 

Respondents’ 

Baseline 

Knowledge 

- Focus Groups 

(3 groups, 

n=27) 

and 

Expert 

Review of 

Mitigation 

Scenario 

- Alternative 

Information 

Treatments 

Stage 2: 

Evaluation 

and Revision -

Group 

Interviews 

(3 groups, 

n=21) 

and 

Individual 

Interviews 

(n=3) - Questionnaire 

Prototype 

Stage 3: 

Pretests -

Individual 

Interviews, 

Debriefing 

(n=29) 

-

Revisio

n

s 

-

Individual 

Interviews, 

Debriefing 

(n=28) 

- Final 

Questionnaire 

Stage 4: 

Internet 

Functionality - Internet 

Pretest and 

Telephone 

Debriefings 

(n=15) - Large-Scale 

Internet Survey 

 

 37
Figure 2. Tabular Choice Format

Wetlands Scorecard #1

How do the Drained and Restored Wetlands Compare?

Wetland Choice #1

Wetland Features Drained

Wetland

Restored

Wetland

Is it marsh, wooded, or a mix of march and woods? Wooded Mixed

How large is it? 14 acres 23 acres

Is it open to public? Yes No

Are there trails and nature signs? No No

How good is the habitat for different species?

Amphibians and reptiles like frogs and turtles Excellent --

Small animals like raccoon, opossum, and fox Good Good

Songbirds like warblers, waxing, and vireo -- Good

Wading birds like sandpiper, heron, or crane -- Good

Wild flowers? Good --

 38
Figure 3. Text Choice Format

Wetlands Scorecard #1

How do the Drained and Restored Wetlands Compare?

Wetland Choice #1

Drained Wetland

The drained wetland is 14 acres in size. It is a wooded wetland. It is open to the

public. It has no trails or nature signs. This wetland is excellent habitat for

amphibians. Small animals such as raccoon, opossum, and fox have good habitat in

this wetland. The habitat is poor for warblers, waxwing, vireo, and other songbirds.

It is poor habitat for wading birds such as cranes, heron, and sandpipers. The growing

conditions for wild flowers are good.

Restored Wetland

The restored wetland is 23 acres in size. It is a mix of marsh and wooded wetland.

It is not open to the public. It has no trails or nature signs. This wetland is poor

habitat for amphibians. Small animals such as racoon, opossum, and fox have good

habitat in this wetland. The habitat is good for warblers, waxwing, vireo, and other

songbirds. It is good habitat for wading birds such as cranes, heron, and sandpipers.

The growing conditions for wild flowers are poor.

 39
Table 1. Respondent Characteristics

Variable Tabular Text Michigan, 

Census 2000 

Households 937 363 3.8 million

Income ($1,000) 54.4 54.1 57.4

Some college 79% 79% 52%

18 to 25 years 8% 8% 9%

Over 65 years 38% 47% 12%

Female 56% 60% 49%

Never visited a wetland 15% 15% -

 40
Table 2. General Properties of the Tabular and Text Mitigation Equation Estimates

Variable Tabulara Texta Difference:

Text-Tabular

No of observations 4685 1811 2874

Correct predictions of yes responses (%) 63 64 -1

Correct predictions of no responses (%) 64 65 -1

Log-likelihood -2814 -1151 --

Cross-sectional effects, standard deviation ( ) 22.3 

(1.64)

49.8

(15.33)

27.5

(15.42)

Respondent effects, standard deviation ( ) 17.0

(1.25)

40.1

(12.37)

23.1

(12.43)

a.. Asymptotic standard errors are given in parentheses.

 41
Table 3. Coefficient Estimates for the Tabular and Text Mitigation Equations

Variable Tabulara Texta Difference:

Text-Tabulara

Acreage of drained wetland 1.42

(0.19)

0.99

(0.68)

-0.42

(0.70)

Change in public access -5.76

(1.01)

-9.76

(4.00)

-3.99

(.412)

Change in wetland type 4.69

(1.14)

1.81

(4.18)

-2.86

(4.33)

Change in poor habitat 

Reptiles/amphibians 8.19

(1.17)

23.46

(8.10)

15.3*

(8.98)

Wading birds 5.76

(1.14)

21.11

(7.53)

15.3 *

(7.62)

Song birds 6.56

(1.16)

21.33

(7.12)

14.8*

(7.21)

Wild flowers 2.33

(1.14)

12.51

(5.41)

10.2*

(5.53)

Change in excellent habitat

Reptiles/amphibians -4.76

(0.76)

1.00

(3.27)

5.8*

(3.35)

Wading birds -5.09

(0.74)

-1.12

(3.19)

4.0*

(3.28)

Song birds -3.80

(0.76)

-1.76

(3.10)

2.0*

(3.19)

Wild flowers -1.94

(0.73)

-3.44

(3.12)

-1.50*

(3.21)

Income ($1,000s) -0.06

(0.02)

-0.03

(0.07)

0.03

(0.06)

Some college -4.25

(1.83)

3.91

(7.49)

8.16

(7.71)

18 to 25 years of age 2.53

(2.70)

3.35

(9.62)

0.82

(10.0)

65 years of age and over 0.41

(3.17)

-3.29

(12.65)

-3.70

(13.05)

 42
Table 3. Coefficient Estimates for the Tabular and Text Mitigation Equations

Variable Tabulara Texta Difference:

Text-Tabulara

Female -2.9

(1.54)

0.59

(5.76)

3.50

(5.96)

Never visited a wetland 8.26

(2.14)

-0.54

(7.92)

-8.80

(8.21)

Intercept 4.75

(2.94)

6.68

(11.27)

1.93

(11.65)

a.. Asymptotic standard errors are given in parentheses. A “*

” indicates that the habitat quality coefficients are

significantly different from zero when evaluated as a group of coefficients.

 43
Discussion

Session VI: Methodological Advances 

in Stated Preference Valuation

Joseph Cooper

Economic Research Service – USDA

US EPA NCEE Workshop

Valuation of Ecological Benefits: Improving the Science 

Behind Policy Decisions”

Washington, DC Oct 26-27, 2004

Overview

I. Specific Comments on:

• Experimental Tests of Provision Rules in Conjoint 

Analysis for Environmental Valuation (Taylor et al.)

II. A USDA Employee’s Perspective on Methodological 

Advances in Stated Preference Techniques

The views presented herein as those of the authors, and do not necessarily represent the views of the Economic 

Research Service or the United States Department of Agriculture.

 44 
I. Comments on Taylor et al.

At least two sources of incentive incompatibility associated 

with conjoint surveys:

2) Incentive incompatibility of referendums with three or more 

choices (or treatments)

1) Incentive incompatibility due to exclusion of an explicit 

provision rule

▫This source of incentive incompatibility is the focus of Taylor 

et al.

▫This is also an issue with dichotomous choice reference 

surveys

I. Comments on Taylor et al.

With regards point 2),

Gibbard-Satterwait Theorem

An election mechanism for 3 or more alternatives 

which is:

 Unanimous

 Strategy proof

is a dictatorship.

Other election methods are not incentive compatible

 45
I. Comments on Taylor et al.

General Question for Conjoint Analysis:

What is the potential response bias associated with 3 

or more alternatives inherent in the voting 

mechanism itself?

I. Comment on Hoehn et al.

Comparison of the tabular choice (fig. 2) to the text 

choice format (fig.3):

• The text choice format has quality rankings of 

“Poor”, “good”, and “excellent.”

• The tabular choice format has quality rankings of 

“--”, “good”, and “excellent.”

• Substituting “--” for “poor” in tabular choice format 

would seem to limit comparability of the two formats.

 46
II. A Governmental Perspective 

How stated preference techniques can be useful to 

ERS

• Measuring the success of conservation programs 

3⁄4 e.g. Conservation Reserve Program (CRP), Environmental 

Quality Incentives Program (EQIP), Conservation Security 

Program (CSP)

• Monetizing the environmental impacts of commodity 

programs

II. A Governmental Perspective 

Source: Smith and Weinberg

Example of relationships we want to measure:

 47
Agricultural Trade Liberalization 

⇓

Change in world prices and quantity

⇓

Changes in production practices, 

input use, and outputs

Changes in physical measures of 

environmental impacts 

⇓

Changes in economic measures of 

environmental impacts

Changes in returns to 

agricultural production

II. A Governmental Perspective

Another Example:

II. A Governmental Perspective 

In stylized fashion,

to evaluate an agr-environmental program, we want to 

be able to approximate dV/dG

where dV/dG = dV/dE · dE/dF · dF/dG

and where V = Nonmarket benefits, G = government 

payments or expenditures, E = environmental 

impacts, F = farm management practices (e.g., 

nitrogen application rates, etc).

 48
II. A Governmental Perspective 

• But in general, CVM survey scenarios are designed 

to produce discrete points in dV/dG or dV/dE , e.g., 

V│G = $ level 1 and V│G = $ level 2

or 

V│E= pollution level 1 and V│E = pollution level 2

• There is probably little one can do to design a CVM 

scenario that approximates dV/dG

II. A Governmental Perspective 

• Hence, the best path is likely to choose CVM 

scenarios to maximize the possibility of achieving a 

statistically significant relationship 

V = f(E,F,G).

• This means choosing a benefit measure V (and 

CVM scenario) that maximizes the possibility of 

obtaining significant relationships between the V 

and policy relevant variables E, F, and G. 

• This suggests considering the available 

environmental process models and data when 

choosing V.

 49
II. A Governmental Perspective 

Examples of notable agri-environmental process models:

• USDA’s Erosion-Productivity Impact Calculator (EPIC)

• USDA’s Soil & Water Assessment Tool (SWAT)

• USGS’s SPAtially Referenced Regressions On 

Watershed Attributes (SPARROW)

II. A Governmental Perspective 

• To make full use of environmental indicators to inform 

decisions, the development and collection of these indicators 

need to be coordinated and integrated with the development 

and collection of behavioral data. 

With economists, we need to stress that

• To make full use of behavorial data to inform decisions, the 

development and collection of these indicators need to be 

coordinated and integrated with the development and 

collection of environmental indicators data.

With physical scientists, we need to stress that

 50
51

Section VI. Methodological Advances in Stated Preference Valuations 

Discussion by T. H. Stevens 

University of MA, Amherst 

I. Embedding in Stated-Preference Methods (Michael Hanemann and Jeff Lazo) 

Although I have been unable to obtain a copy of this paper, I found Hanemann's 

presentation both stimulating and useful. Of particular importance is the notion that some types 

of embedding, such as geographical scope effects, can often result from rational, well informed 

decision-making. For example, many respondents may express the same value for preserving a 

nearby wetland as they do for preserving all wetlands in a larger region simply because the 

nearby wetland is the only one that is really important to them. Many other logical reasons for 

scope effects were outlined in this presentation which suggests that (a) it is important to examine 

psychological factors that might influence respondent's decision making and (b) debriefing 

should be an important component of the stated preference methodology. Many important issues 

remain to be addressed, including definition of the relevant choice set for valuation of public 

goods. 

II. Experimental Tests of Provision Rules in Conjoint Analyses for Environmental Valuation 

(Taylor, Boyle, Morrison). 

 This paper focuses on a very important issue. Conjoint (choice) analyses is being used 

widely, but little is known about potential biases that might be associated with this technique. In 

particular, since provision rules are generally not incorporated within the conjoint format, this 

method might produce inaccurate results. 

 The experiments involving hypothetical payments conducted by the authors suggest that: 

 1. Respondents were more likely to "purchase" a private good when the conjoint 

question did not include an incentive compatible provision rule. 
52

 2. Subjects were less responsive to the price of a public good when the conjoint 

question did not include an incentive compatible provision rule. 

 Taken together, these findings imply that results derived from the traditional conjoint 

approach (without provision rule) are likely biased upward. 

 Experiments involving real payments were also conducted, but these results were not 

available prior to Taylor's presentation. However, the presentation seemed to suggest that 

explicitly stated provision rules reduced hypothetical bias associated with the conjoint analyses. 

If so, then it is very important to incorporate appropriate provision rules in conjoint analyses. 

 It is important to note, however, that comparisons were not made between an incentive 

compatible CV format and an incentive compatible conjoint format. Such a comparison is 

important because CV and conjoint techniques differ in several respects other than the provision 

rule. That is, even if conjoint methods are modified to incorporate appropriate provision rules, 

conjoint and CV results may still diverge because of other differences between these formats. 

For example: 

 (1) Substitutes are made explicit in the conjoint (CJ) format and this may encourage 

respondents to explore their preferences and tradeoffs in more detail. Indeed, as noted by Gan 

and Luzar (1993), conjoint analysis 'can be characterized as an extension of the referendum 

closed-end CV method in which large numbers of attributes and levels can be included in the 

analysis without overwhelming the respondents' (p. 37). As shown by Boxall, et al. (1996), 

when compared to CJ, CV results may therefore be biased upward because respondents to the 

'typical' CV survey are usually asked to consider fewer substitutes. 

 (2) From a psychological perspective, the process of making choices in the CJ format 

may be quite different from that associated with making decisions about WTP (Irwin, et al., 
53

1993; McKenzie, 1993). That is, respondents may react differently when choosing among 

commodities that have an assigned price as compared to making dollar valuations of the same 

commodities. Moreover, Irwin, et al. (1993) found that CV questions lead to relatively greater 

preference for improved commodities, such as TVs and VCRs, while choice questions yielded 

relatively greater preference for environmental amenities like air quality. Similar results were 

reported by Brown (1984). Irwin, et al. (1993) concluded that if monetary prices are an attribute, 

they carry more weight in determining a response measured in dollars (e.g. CV) than they do in 

determining a rating or choice response. This arises from the fact that choices seem to be driven 

from reason and arguments to a greater extent than are pricing responses. 

 (3) CJ respondents can express ambivalence or indifference directly. As a result, CJ 

surveys may result in relatively less non-response and protest behavior. Moreover, allowing for 

respondent uncertainty may have a significant effect on the WTP of those who do respond. For 

example, Ready, et al. (1995) compared a dichotomous choice CV format to a polychotomous 

choice format. Their CV question asked respondents to determine whether or not they preferred 

a given program while the polychotomous choice format gave six options (i.e., definitely prefer, 

probably prefer, maybe prefer, maybe not prefer, probably not prefer, definitely not prefer). This 

format was motivated by the belief that respondents might be more comfortable answering 

valuation questions when given the opportunity to express strength of conviction; since the 

polychotomous method allows for a range of answers, it might produce a more accurate 

description of respondents' preferences. In two empirical studies, preservation of wetlands and 

horse farms, the polychotomous format yielded a higher rate of usable responses and much 

higher WTP estimates. 
54

 More recently, Champ, et al. (1997) found that although contingent values were greater 

than actual donations for an environmental good, when the contingent values were restricted to 

respondents who said they were very certain to contribute, mean CV and actual donations were 

not statistically different. Ekstrand and Loomis (1997), Alberini, et al. (1997) and Wang (1997) 

also found that contingent value estimates vary widely depending on how respondent uncertainty 

is incorporated in the analysis. 

 In summary, conjoint (choice) and CV formats differ in several ways, and correction for 

provision rule may not resolve many of the differences between traditional CV and conjoint 

estimates. 

III. Stated-Choice Experiments to Estimate In-Kind Values for Ecosystem Mitigation (John 

 Hoehn, Frank Lupi, Michael Kaplowitz). 

 This paper addresses another very important issue--do respondents suffer from 

information overload in the stated choice format, and if so, what are the consequences and what 

can be done about this potential problem? 

 The authors use a split sample approach to compare text and tabular information formats 

with the result that the tabular presentation was successful in reducing information complexity 

and information overload. 

 Specific comments are as follows: 

 1. An internet survey that produced an 8 percent response rate was used in this 

study. Much more information is needed with respect to non-respondents. 

 2. Another interesting research question would be whether the differences observed 

in this study are also found in mail surveys. 
55

 3. In this study, text respondents tended to exhibit loss aversion while tabular 

respondents did not. But if loss aversion is part of "human behavior", elimination of loss 

aversion might produce biased results. So, in this sense is a tabular format really "better" than a 

text version? 
56

References 

Alberini, A., Boyle, K., Welsh, M., 1997. Using multiple-bounded questions to incorporate 

preference uncertainty in non-market valuation. In: Englin, J. (Compiler), Tenth Interim 

Report, W-133 Benefits and Costs Transfer in Natural Resource Planning, University of 

Reno, Reno, NV. 

Boxall, P., Adamowicz, W., Swait, J., Williams, M., Laviere, J., 1996. A comparison of stated 

preference methods for environmental valuation. Ecol. Econom. 18, 243-253. 

Brown, T., 1984. The concept of value in resource allocation. Land Econom. 60, 231-246. 

Champ, P., Bishop, R., Brown, T., McCollum, D., 1997. Using donation mechanisms to value 

nonuse benefits from public goods. J. Environ. Econom. Manag. 32 (2), 151-162. 

Ekstrand, E., Loomis, J., 1997. Estimated willingness to pay for protecting critical habitat for 

threatened and endangered fish with respondent uncertainty. In: Englin, J. (Compiler), Tenth 

Interim Report, W-133 Benefits and Costs Transfer in Natural Resource Planning, University 

of Reno, Reno, NV. 

Gan, C., Luzar, E.J., 1993. A conjoint analysis of waterfowl hunting in Louisiana. J. Agric. 

Appl. Econom. 25, 36-45. 

Irwin, J.R., Slovic, P., Licktenstein, L., McClelland, G., 1993. Preference reversals and the 

measurement of environmental values. J. Rick Uncertrain. 6, 5-18. 

McKenzie, J., 1993. A comparison of contingent preference models. Am. J. Agric. Econom. 

75, 593-603. 

Ready, R., Whitehead, J., Blomquist, G., 1995. Contingent valuation when respondents are 

ambivalent. J. Environ. Econom. Manag., 29 (2), 181-197. 

Wang, H., 1997. Treatment of don't know? Responses in contingent valuation survey: a random 

valuation model. J. Environ. Econom. Manag. 32, 219-232. 

 
57

Summary of the Q&A Discussion Following Session VI

Elizabeth David (Stratus Consulting, Inc.) 

Dr. David introduced herself by saying that prior to working for Stratus she worked for a 

number of years with the State of Wisconsin’s Department of Natural Resources, and she 

stated that she would “feel much better” if she knew that John Hoehn had “familiarity 

with questionnaire design.” She cited the University of Michigan’s Survey Research 

Center within the Institute of Social Research, where there is a “whole section that 

worries about how to design questions.” Acknowledging that the wetlands problem is 

particularly difficult and extremely complicated with “so many services associated with 

it,” she wished that Dr. Hoehn had “grounded his questions about wetlands in the existing 

literature about how to deal with a very complicated set of interactions.” 

John Hoehn (Michigan State University) 

Dr. Hoehn responded by stating, “We actually did work with a number of people from 

the University of Michigan as consultants on the project. We certainly did try to access 

that knowledge base.” 

________________________ 

Joel Huber, (Duke University) 

Dr. Huber commented, “I wanted to talk to the issue of whether we’re trying to get from 

people the best answer or their first answer. There’s a certain notion that there’s true 

utility up there and we need to ask them without them thinking about it too much and get 

it out. I actually would have gone the other way. If you think about it, what you really 

want to do is not get what they would do quickly but what they would do if they thought

about it. Because here we’re talking about policy, and most of these are rich issues and 

they’re deep issues. So, I really applaud, John, what you’re doing in terms of trying to 

simplify and trying to actually test. . . . This can be sort of discouraging, because much of 

the economics becomes harder to apply—the appearance of the answer depends on how 

you ask the question, and knowing that puts you in a type of limbo because there are all 

kinds of skills needed that you don’t normally have. Typically in your work, though, you 

have options of doing different versions, so I would suggest taking advantage of this. 

You can have version A and version B and you don’t necessarily need to mention it—if 

they work out the same, great. What you’re aiming for is what a person would do if they 

thought about it a lot—it’s quite different from what I would call the sort of implicit 

utility.” 

Michael Kaplowitz (Michigan State University) 

Dr. Kaplowitz said, “I just want to make one comment, because a lot of this discussion 

the last two days has been on economics and ecology, and the work that John (Hoehn), 

Frank (Lupi), and I are doing—and the work that I think many people here have done—is 

really work that spans economics and survey research. For example, I never would have 

thought that we would be publishing in survey research outlets, but our four-step design 

process is now something that survey researchers are using or are thinking about using in 

their work in other fields. So, I think there’s a lot of crossover and lessons we can learn.” 
58

When asked by John Hoehn what book was forthcoming on this topic, Dr. Kaplowitz 

replied, “Stanley Presser and a bunch of people have a questionnaire development and 

evaluation text being published by Wiley Press.” 

________________________ 

Elena Besedin (Abt Associates,Inc.)

Addressing Michael Hanemann, Ms. Besedin commented that Dr. Hanemann, in 

reporting on his study, had mentioned an internal scope test in which the survey 

participants indicated that they held widely disparate views as to how many birds would 

represent a “significant” effect on bird population. She raised the question of whether 

this really represents a scope issue “compared to, for example, a background information 

issue.” She noted that “sometimes scientists have difficulties measuring bird and other 

wildlife populations.” Ms. Besedin added that you “can’t really judge whether an effect 

is large or small” without having some idea of the total population figure. She concluded 

by asking Dr. Hanemann whether this information (the total population figure) was 

available to the focus group who gave the noted response, and if so, how would he 

explain their conclusions. 

Michael Hanemann (University of California-Berkeley) 

Dr. Hanemann responded by saying that “the survey gave famously ambivalent 

information, because it said: 2,000 birds or under 1%; 20,000 birds or about 1%; and 

200,000 birds or under 2%. From one perspective, these are all speaking of 1 or 2%--

little to no difference. . . . I interpret this as saying that, in fact, people were looking at the 

percentages, and there’s abundant literature from Slovik and others that indicates that 

percentages are what people think of. The difference between under 1% and under 2% is 

unimportant, and I think the attitude question about what this means to the population 

suggests that they were looking at the percentages, and so they had a real basis. 

Obviously, if they were looking at the numbers, that’s striking, but it seems they were 

focusing on the percentages, and the differences are unimportant.” 

________________________ 

John Hoehn 

Dr. Hoehn referred to Joel Huber’s comment that the decision they were trying to get at is 

one that a person would make if they had a little more time to assimilate information and 

to think about it. He said, “I think that is certainly a target. This format problem is 

trying to make the assimilation task easier for respondents, so they don’t have to spend as 

much time on assimilation and can put more time into the decision and focus on that 

problem. . . . It is a difficult problem, and some of the work we’re doing is contributing to 

that literature on survey design . . . because these are different sorts of questions than 

asking, “Who are you going to vote for for president?”—and even there you see a lot of 

variability these days. You know, we are asking people to make difficult choices when 

we address wetlands issues—they’re a distinct kind of problem in terms of survey design 

and the issues they raise with respect to human cognitions.” 
59

Michael Hanemann 

Dr. Hanemann added, “There’s a different strategy, in sociology at least, in attitude 

measurement. The strategy in attitude measurement is to ask a large battery of questions 

and then to reduce them. I see this relating to the Lancaster Model. If, by an attitude, 

you mean something broad, such as patriotism or law-and-order, then it makes sense that 

there’s a large number of questions that would touch on that, and you could average 

them. . . . So, when you’re measuring something very broad, then it’s possible to have a 

large number of imprecise measurements.” 

Joel Huber 

Dr. Huber followed with these comments: “If you’re trying to get attitudes, what you 

want is quickness—that is, you want to see how a person reacts to a certain “picture.” 

That is often mediated by thought. . . . So, some things are attitude questions, but the 

tradeoffs are what I’d call “rational.” They’re very different modalities—one is fast and 

the other is slow, and you’re actually overcoming your initial thoughts. So, depending on 

what you want to do, you’d go one way or the other.” 

________________________ 

END OF SESSION VI Q&A