Incentive and Informational Properties of Preference Questions 1

Richard T. Carson

Theodore Groves

Mark J. Machina

Department of Economics, 0508

University of California, San Diego

La Jolla, CA 92093

rcarson@ucsd.edu

Draft

February 2000

 

1Earlier versions of this paper were presented in Oslo as a plenary address to the European Association of

Environmental and Resource Economics, as an invited paper at the Japanese Form on Environmental Valuation

meeting in Kobe, and at a NOAA conference on stated preference methods. Support of U.S. Environmental

Protection Agency cooperative agreement R-824698 in carrying out the research reported on in this paper is

gratefully acknowledged. The views expressed are those of the authors and not necessarily those of the U.S.

Environmental Protection Agency.
1

Introduction

Businesses and governments frequently use surveys to help determine the relevant

public’s preferences toward different actions that could be taken. Applications are particularly

common in environmental valuation (Mitchell and Carson, 1989), health care (McDowell and

Newell, 1996), marketing (Louviere, 1994), political science (King, 1989) and transportation

(Hensher, 1994). As long as the economic agents (hereafter, “agents”) being surveyed believe

that the survey responses might influence actions taken by businesses and governments

(hereafter, “agency”), the standard economic framework suggests that the agents should respond

to the survey in such a way as to maximize expected utility.

Given the billions of dollars spent annually on surveying and the frequently voiced

concern that marketing surveys determine the fate of products and that major political decisions

are largely poll-driven, the position of many economists that survey responses should be ignored

as a source of information on preferences is somewhat surprising. These economists seem to

regard survey responses as either completely meaningless because they are answers to

hypothetical questions or as completely useless because agents will respond strategically. The

first reason violates the standard rationality condition assumed of agents if agents believe that

agency decisions are being made at least in part on the basis of the survey responses. The second

reason stops short of the more relevant question: what are the strategic incentives and how

should they influence responses?

In this paper, we systematically explore implications of the economic maximization

framework for the behavior that one should expect to see from rational agents answering survey

questions concerning preferences. The economic literature on neoclassical choice theory and

mechanism design (Hurwitz, 1986; Groves, Radner and Reiter, 1987; Varian, 1992) provides the
2

theoretical foundation for our work. This body of work can be contrasted with those who reject

this framework in favor of other psychologically based theories (e.g., Kahneman, Slovic and

Tversky, 1982; Sugden, 1999, McFadden, 1999). We believe that at least some of the evidence

put forward in favor of those theories, particularly with respect to what differences should be

expected with respect to asking questions using different response modes, has been incorrectly

interpreted. We have endeavored here to put forth our results in an intuitive non-mathematical

fashion as we hope fundamentally that our models represent a common sense approach to

thinking about how agents should view preference questions. In the model informally presented

here, agents are assumed to decide (1) whether they care about how the outcome might be

influenced by the answers they provided, (2) whether the aspects of the scenario described are

plausible, and (3) how the survey results are likely to be used. Judgements respecting these

assumptions need not be elaborately or explicitly articulated any more than most judgements in

life are. These three assumptions combined with the basic maximizing rationality assumption are

capable of yielding a surprisingly rich picture of the manner in which agents should respond to

survey questions.

A major reason that estimates of economic value from surveys are looked upon with

suspicion by economists is a number of results that seem inconsistent with respect to economic

intuition. These anomalous results have been interpreted by different researchers as evidence of

(a) the hypothetical nature of the question, (b) strategic behavior2, or (c) preferences that are

 

2

 The possibility of strategic misrepresentation of preferences has long been seen as one of the central problems in

public economics. Samuelson (1954) argued: “It is in the selfish interest of each person to give false signals to

pretend to have less interest in a given collective activity than he really has.” He made specific reference to the

possibility of strategic behavior with respect to the use of surveys. Samuelson’s admonition, repeated in many

textbook discussions of public goods, had a profound effect on how many economists view the survey questions.

The mistaken inference made by many from this admonition was to equate strategic behavior with lying. As the term

is used in the modern mechanism design literature in economics, strategic behavior is merely synonymous with a

rational agent maximizing (broadly defined) self-interest. Mechanism design theory has shown that the optimal

strategic behavior for agents in many instances is to truthfully reveal their preferences. Whether this is the case or
3

either ill defined or inconsistent with economic theory. In attempting to systematically categorize

these anomalies it becomes immediately apparent that there is an antecedent question: does a

survey question need to meet certain conditions before it should be expected to produce results

that are consistent with economic theory?

This question turns out to be relatively easy to address from the standpoint of economic

theory. First, the agent answering a preference survey question must perceive responses to the

survey question as potentially influencing agency action. Second, the agent needs to care about

what the outcome of that action is.3 We will term surveys that meet these two basic criteria as

consequential survey questions and those that don’t as inconsequential survey questions. In more

formal terms, we can state the following:

Consequential and Inconsequential Preference Survey Questions:

A. If the survey results are seen by the agent as potentially influencing agency actions and the

agent cares about the outcome of that action, then the agent should treat the survey question as an

opportunity to influence those actions. In the case of a consequential survey question, standard

economic theory applies and the response to the question should be interpretable using

mechanism design theory concerning incentive structures.

B. If the survey responses are not seen as having any influence on agency decisions or the agent

is indifferent to all possible outcomes of the agency decision, then all possible responses by the

agent will be perceived as having the same influence on the final outcome. In this case of an

inconsequential survey question, economic theory makes no predictions about the nature of the

responses to the survey given by the agent.

 

not depends upon the particular format of the preference question asked and other aspects of the scenario, including

the type of good involved.

3

 For instance, a non-smoker may not care about the addition of a new type of cigarette with a much lower nicotine

level and a higher price to the current cigarette choice set. Confusion often exists over the magnitude of the possible

change in utility from agency action and the incentives the agent faces in the response given to the question. The

size of the utility change generally does not influence the incentive structure of the question as long as there are

differences in utility levels between different agency actions. The size of the utility change can influence agent

participation in the survey.
4

Most preference survey questions asked by businesses and governments meet the two

basic criteria for being a consequential survey question, and hence, should be interpretable in

economic terms.4 There are, however, many preference survey questions that do not meet these

criteria. While most of these inconsequential survey questions could be characterized as issuing

from psychology lab exercises with undergraduates, there are plenty of real world examples.5 It

is pointless to try to explain apparent economic anomalies in inconsequential survey questions

because any response to such a question has the same effect on the agent’s utility. We are

formally rejecting the notion sometimes advanced by proponents of the use of preference survey

questions, which if a respondent perceives no gain or loss from how a preference survey is

answered then that respondent will truthfully answer the question. While such an assumption

may indeed be true, there is no basis in economic theory to either support or deny it.

Among questions meeting these two criteria for being consequential to the agent, we

examine five key issues which should illustrate both the power and limitations of economic

theory to explain a large body of empirical evidence related to the performance of survey

questions under particular conditions. First, we look at the properties of binary discrete choice

questions under different circumstances. In particular, we examine whether binary discrete

choice questions are incentive compatible in the sense that truthful preference revelation

represents an optimal (and the dominant) strategy for the agent. The empirical evidence suggests

that such questions often work well: they predict actual behavior quite closely and they are

 

4

 Marketing research firms, in particular, face a constant battle between asking questions to only those who are

currently using a product category and trying to reach the larger and harder to identify population of all potential

users. For public goods provided via taxation, the situation is generally easier. Even if the respondent does not care

whether the good is provided at zero cost, the respondent does care about its provision if the tax cost is positive.

5

 Inconsequential preference questions can most often be identified by having one or more of the following

identifying characteristics: (a) being asked of a population or at a location that is unlikely from the perspective of an

agency seeking input on a decision, (b) providing few if any details about the goods and how they would be

provided, (c) asking about goods that are implausible to provide, and (d) providing prices for the goods that are

implausible.
5

sensitive to factors such as the scope of the good being valued. However, there are instances

where such questions perform quite badly. Second, we consider the reasons responses to

repeated binary discrete choice questions (e.g., double-bounded dichotomous choice) by the

same respondent are often inconsistent with each other. We also consider what information

might be provided to the agent by the second choice question in this section. Third, we look at

whether binary discrete choice questions and open-ended continuous response questions should

produce similar estimates of statistics such as mean or median willingness to pay (WTP). In this

section, we pay particular attention to the issue of what role, if any, information on cost should

have on reported WTP values. Fourth, we consider the implication of moving from valuation of a

single good to valuation of multiple goods, first in the context of the sequence of pair

comparisons and then in the context of the increasingly popular multinomial choice questions.

To begin to understand these issues, it is necessary to first confront what we have termed the face

value dilemma.

The Face Value Dilemma

Economists tend to either reject preference survey results out of hand or treat the answers

as truthful responses to the question asked. We term this latter behavior as taking the survey

answers at face value. The two positions are not unrelated as both are result-oriented rather than

process-oriented; many economists who reject the use of survey questions do so because the

results are anomalous if taken at face value. We believe that either rejecting the usefulness of the

preference survey answers or taking them at face value is likely to be wrong in many

circumstances even when the two basic criteria for a consequential preference survey question

have been met.
6

The face value assumption can be formally defined as “the assumption that respondents

always truthfully answered the specific preference question intended to be asked”. There are two

key parts of this assumption: (a) respondents always truthfully reveal preferences, and (b) the

specific question being asked is the one being answered. Note that (a) and (b) are both very

strong assumptions. While the mainstream economic position is that (a) is dubious due to the

strategic behavior, this assumption is routinely maintained in marketing research, political

polling, psychology, sociology and other fields heavily dependent on survey research. In

contrast, while economists who do use survey results routinely seem to believe (b), survey

researchers have shown this to be a dubious assumption (Sudman, Bradburn, Schwarz. 1996).

Interpreting responses to survey questions appropriately requires consideration of the

possibility that neither part of the face value assumption maybe be true. For truthfully revealing

preferences, objections that agents may be responding strategically are insufficient to reject the

use of consequential preference survey questions, as it may be in the respondent’s strategic

interest to truthfully reveal their preferences under some question formats in particular contexts.6

With respect to the decoupling of question and answer, the survey research community’s

usual rationale for the possibility that respondents may answer a different question than the one

asked, is simply that respondents may not understand the question actually asked and instead

answer the question that they think is being asked. Part of the survey designer’s art lies in the

crafting of language that elicits the answer to the question that the researcher intended to ask

(Payne, 1951). From the perspective of preference survey questions for non-marketed goods or

new consumer products, this issue needs to be taken particularly seriously since the development

 

6 Furthermore, under other question formats, the expected direction of the bias in responses can be theoretically

predicted in some instances and empirically confirmed. In such cases, the results, even if biased, may be useful and

often sufficient for agency decision making (Hoehn and Randall, 1987).
7

of questionnaires describing such goods is among the more difficult of survey design tasks; and

most economists developing such surveys have little formal training in survey design. The preeminent issue here is that if survey responses are to be taken at face value, the question as written

should elicit the answer to the question intended to be answered by the designer with all the

conditions with which the designer wanted it answered. If this does not happen the results can

easily be taken as implying violations of economic theory, when what has in reality happened is

that agents have answered a different question. 7

A further issue should be raised which concerns asking preference questions with implausible

premises, for example, asking a binary discrete choice question with an implausibly high or low

cost for providing the good. In such instances, respondents are likely to substitute what they

consider to be the expected cost for the good and answer on that basis. Another easily recognized

variant of this issue concerns implausible characteristics of the good provided, such as an

assertion that a risk reduction program would be 100% successful, an assertion which is likely to

be discounted by agents. More complicated variants of the issue manifest themselves when a

respondent is given information at various points in a survey that is inconsistent. Examples

include providing two different cost numbers in the double-bounded dichotomous-choice

elicitation format and asking respondents about the provision of different levels of the same

public good at different places in the survey. A key implication of this line of argument is that

 

7

 For example, if a subset of agents does not believe that the good can be provided in any amount, these agents

should be insensitive to the quantity (scope) of the good to be provided even though they may place a positive value

on it. Divergences between the intended and answered question will always occur to a greater or lesser degree. The

survey designer should endeavor to minimize them and the analyst should determine how they need be taken into

account in order to arrive at reliable estimates. It should be noted that there is nothing unique about the use of stated

preference data with respect to this issue. Most economic analyses of revealed preference data use objective

indicators of good attributes to predict agent choices even though using agent perceptions of them can usually be

shown to provide better predictions (Adamowicz et al., 1997). Estimates of the value of a statistical life based upon

hedonic wage equations have always been plagued by the need to make the demonstrably false assumption that

agents were aware of the objective risk level used as a predictor variable in the equation.
8

there are likely to be limits to the range of preference questions that a researcher can expect to

have respondents answer. Survey questions can extend the range of goods and their attributes,

including price, considerably beyond what agents have previously experienced; but any

counterfactual scenarios must be credible portraits of possible future outcomes.

A Simple Typology of Elicitation Formats

The truthful preference revelation part of the face value assumption implies different

conditions for different elicitation formats. This can most easily be seen by noting, that from a

strategic perspective, all of the standard question formats can be shown to be generalizations of

the single binary discrete choice format (Figure 1). Under this format, the respondent is told

about two different alternatives and is assumed to pick which of the two alternatives provides the

highest level of utility. As we discuss at length below, this assumption may be justified under

some sets of conditions but not others.

FIGURE 1

Single Binary Choice

“One-Shot Choice”

Sequence of Binary Choices

*Assumes Independence

Across Choices

Multinomial Choice

One-Shot Choice,

k > 2 Alternatives

Equivalency (Valuation)

* Assumes a Continuous

 Variable (e.g. money)

Sequence of Multinomial

Choices

* Assumes ‘Independence’

Across Choice Sets

Sequence of Intensity-ofPreference Questions

*Assumes Cardinal Utility
9

There are three basic ways a single binary discrete choice question can be generalized. These

are the open-ended matching type question, a sequence of binary choice choices, and the

multinomial choice question. Matching questions, rather than ask for a choice between two

alternatives, drops an attribute level (typically cost) of the second choice and asks the agent to

provide the quantity of the attribute level that would make the agent see the two choices as

equivalent in terms of utility to the agent. A sequence of binary choice questions adds the

assumption that the agent answers each pair of choice independently to the assumption made of

the single binary discrete choice. A number of different formats can be shown to be strategically

manifestations of the sequence of binary discrete choice questions including the popular double

bounded dichotomous choice format in contingent valuation (Hanemann, Loomis, and Kanninen,

1991) and the complete ranking of alternatives popular in marketing which can always be

exploded into a set of binary paired comparisons if the independence assumption holds

(Chapman and Staelin, 1982). Another commonly used variant of the sequence of binary choices

asks agents to “rate” one choice relative to the other on a numeric scale (e.g., 1 to 10) and

exploits the information revealed about preference intensity (Johnson and Desvousges, 1997).

This adds the assumption of cardinal utility. A multinomial choice question adds the assumption

that the agent picks the most preferred out of k > 2 alternatives. A popular variant of this format,

a sequence of multinomial choice sets (Louviere, 1994) adds the same assumption that a

sequence of binary choice questions does, independence in responses across the choice sets.

For each of these questions formats it is possible to look at the divergence between the face

value response and the strategic response. It is also possible to look at differences in the set of

information conveyed by a particular elicitation format. Because the different elicitation formats

are generalizations of the binary discrete choice format and because it can be shown that the
10

binary discrete choice format has different strategic properties in different context we start with

an examination of that format.

Binary Discrete Choice Preference Questions

A single binary discrete choice question between two alternatives, one typically being the

status quo, is one of the most commonly used preference elicitation formats. It has a long history

of use in survey research, and most other discrete choice and ranking formats can be easily

shown to be generalizations of it. Bishop and Heberlein (1979) showed that this format could be

used along with the random assignment of respondents to different monetary costs to recover the

distribution of willingness to pay or willingness to accept compensation (WTA). Later papers by

Hanemann (1984a, 1984b) formally worked out the utility theoretic approach from a random

utility perspective (McFadden, 1974); and Cameron (1988) provided a purely statistical approach

of tracing out the latent (unobserved) WTP or WTA variable in a manner similar to dose

response experiments in biology or medicine. McConnell (1990), Kriström (1997), Haab and

McConnell (1997; 1998) and Hanemann and Kanninen (1999) provide comprehensive

examinations of the statistical issues involved in using the binary discrete choice format. While

we will generally ignore the substantive issues raised in these papers with respect to the

estimation process, we do note some of the implausible estimates that in the literature appear to

be the result of failing to adequately model the data and incorporate sensible restrictions implied

by economic theory.

Much of the attention focused on the binary discrete choice elicitation format in recent

years is due to the NOAA Panel on Contingent Valuation’s (Arrow et al., 1993) recommendation

for its use as a consequence of its well-known property of being incentive compatible in some

circumstances. Indeed, one of the core results in mechanism design theory independently derived
11

by Gibbard (1973) and Satterthwaite (1975) is that no response format that allows for more than

a binary response can be incentive compatible without assuming restrictions on the realm of

allowable agent preferences.

However, the Gibbard-Satterthwaite result is essentially a negative one—no response

format with greater than a binary choice (including all multinomial and continuous response

formats) can be incentive compatible without restrictions on preferences. This result does not say

that all or even any binary discrete choice formats are incentive compatible; only that this is the

only response format that is potentially incentive compatible.

It has long been known that in some settings that the binary discrete choice format is

incentive compatible (Farquharson, 1969). The best-known examples are political races with

only two candidates and binding (approve/disapprove) referendums with a plurality (usually

majority or two-thirds approval) vote requirement. The binding referendum is a useful departure

point for our discussion and the NOAA Panel references this mechanism before their

recommendation to use a binary discrete choice format in contingent valuation (CV) surveys.

The first question is whether it is the binding nature of the referendum that makes it

incentive compatible. Carson, Groves, and Machina (1997) consider an advisory referendum

vote.8 They show that replacing the binding plurality vote requirement with the weaker

assumption that, over some range, the government is more likely to undertake an action the

larger the percentage in favor.9 The plurality vote requirement is a special case of this

 

8

 Many well-known referendums are technically advisory referendum. For example, Norway’s vote on whether to

join the European Union (EU) was an advisory referendum. Some observers believed that if the vote in favor were

only a very slim majority, that the government would not join the EU, however, if a substantial majority favored

joining then the government would join the EU.

9

 It is necessary to assume that agents believe they have only influence locally around the amount they are asked if

this response function is considered to cover the case where the amounts agreed to are summed. We are indebted to

Pere Riera for this observation.
12

assumption with the knifed-edged decision rule that any vote less than the required plurality for

the new (“yes”) alternative results in the default (“no”) alternative being implemented.

The second question is: does substituting an advisory survey for an advisory referendum

alters the incentive properties of the mechanism? Green and Laffont (1978) have shown that any

economic mechanism of the types being considered in this paper can be implemented using a

sampling approach rather than complete participation. Thus, we come to the following:

Result: It is possible to replace the binding nature of an incentive compatible referendum

with the more general assumption that the agency is more likely to undertake the action

the higher percent in favor. It is also possible to substitute a survey of the public for a

vote of the public on the issue. Neither of these changes, alone or together alter the

original incentive structure of the binding referendum.

A small number of CV studies (e.g., Carson, Hanemann, and Mitchell, 1987; Polasky,

Gainutdinova and Kerkvliet, 1996), which have compared survey estimates to the vote on actual

binding referendums, have found the two to be quite close. A very large body of evidence from

polling on referendum suggests that surveys taken close to an election generally provide quite

good predictions of actual referendum votes.10 It is important to note, however, that it is not

casting the preference question as a referendum that provides its desirable incentive properties. It

is the cast of the preference question in terms of being able to influence a government decision

with a binary favor/not favor format.

Two key assumptions have been made in the discussion of the preceding sequence of

mechanisms. The first assumption is that the agency (i.e., government) can compel payment for a

 

10 Predicting an actual election vote from a survey involves two key difficulties unrelated to whether agents

truthfully reveal their preferences in surveys. The first is that the information set the voter uses on election day may

have changed from the one at the time of the survey due to activities such as political advertising and media

coverage. It is this factor that makes surveys taken close to an election generally more accurate than surveys taken at

some distance from the election. (The dynamics of the information process are such that the proponents of the

measure are usually able to initially put out a largely unopposed positive message. As opponents slowly start their

negative campaign, support for the measure falls over time.) The second is predicting who is going to actually vote.

The characteristics of a good random sample of the public may be substantially different from the characteristics of

the sample of the public that actually votes.
13

good if provided. The second assumption is that only a single issue is involved. Relaxing the first

assumption destroys the incentive properties of what we will call the referendum—advisory

referendum—advisory survey (RARAS) mechanism. To see this, consider the case where a

charitable organization wants to provide a public good via voluntary contributions. A “yes”

response to a binary discrete choice survey question of the form: “would you contribute $X to a

fund to purchase the specified good if we started the fund?” will encourage the charitable

organization to undertake the fundraising effort. Upon mounting the fundraising effort, the

optimal strategic response of an agent who wants the public good will be to contribute less than

her maximum willingness to pay for the good and in many instances to contribute nothing. 11 The

preferred strategy is to sit back and wait to see if the good is provided without her contribution.

This is the classic free riding behavior which economists have long seen as perhaps the

fundamental problem with the provision of public goods. What is interesting in this case is that

the same incentive structure which should cause free riding with respect to the actual

contributions should induce respondents in a survey to over pledge because doing so helps to

obtain the later opportunity to free ride. A number of empirical studies confirm the large

predicted divergence between survey-based predictions of contributions and actual contributions

(e.g., Seip and Strand, 1992; Champ et al., 1997).

Switching to the case of introducing a new private good does not improve the incentive

situation. As long as there is any positive probability of wanting the new good at the stated price,

the respondent should say, “yes—would purchase.” The agent’s logic is that such a response will

encourage the company to produce the good, with the agent being able to decide later whether to

 

11 In many charitable fundraising efforts, the quantity of the good provided is increasing in the amount of money

raised. In such a case, it may be optimal for a (non-pivotal) agent who desires the good to contribute at a positive

amount toward its provision (Blume, Bergstrom and Varian, 1986).
14

purchase. Since increasing the agent’s choice set in a desirable way increases utility, the optimal

response is “yes.” Folk wisdom from the marketing research literature supports the notion that

consumers overstate their purchase proclivities for new products (Greenhalgh, 1986). Evidence

from experiments in economics (Cummings, Harrison, and Rustöm, 1995; Johannesson, Liljas,

and Johansson, 1998) also supports this conclusion. The marketing research approach has tended

to either shift to a different measurement scale such as the probability of purchasing (Inforsino,

1986) or to ask about more than one good (Louviere, 1994).

There is some irony in this result as it has so often been said that if standard CV

elicitation formats did not work well for private goods then they would not work for pure public

goods that are not bought and sold in the marketplace. The familiarity argument that is so often

heard in support of doing experiments with private goods to learn about how CV is likely to

work in the best case scenario (Neil et al., 1994) is misguided. Examined in this light, the

introduction of a new private good is one of the worst-case scenarios for a binary discrete choice

question. It should not be surprising that the binary discrete choice format, which while initially

seeing usage in marketing research, is now rarely used.

The ability of the agency to coercively collect payment for the good is the property that

causes the agent to try to influence the agency’s decision in the desired direction taking account

of both the cost and the benefits of the action to the agent.12 Voluntary contributions allow for

 

12 It is interesting to ask whether it is the two-step nature of a survey followed by a contribution/purchase that leads

to the survey question not being incentive compatible. The answer is no. Consider the situation whereby the only

way a public good can be provided is if it obtains the requisite plurality vote in a referendum and the legislature gets

to decide whether to put the issue on the ballot for a vote. The legislature does not want to waste the public’s time

putting on propositions to vote on if they stand little chance of passing. The legislature (or the measure supporters)

commissions a survey to determine the likely fraction of the public that would vote in favor of the measure. The

only consistent responses (given no change in the information set) to the survey and actual referendum vote are

“yes” to both the survey and the referendum or “no” to both the survey and referendum. For those in favor of the

measure, the only way to get the good is to get the referendum put on the ballot and have the measure passed. “Yes”

responses to both opportunities increase the chance of both. For those opposed to the measure, saying “yes” to the

survey increases the chance that it will get put on the ballot, which in turn increases the chance that the agent will

have to pay for the good, even though the good is not worth the cost to the agent if provided.
15

the possibility that the survey response encourages the fund-raising effort to be undertaken, and

hence, the possibility of free riding during the actual fund-raising effort. Thus, agents who want

the good provided should say “yes” (would contribute) to the survey. In turn, it will be optimal

for some of these agents to free ride in the expectation that other agents would contribute enough

to provide the good. In this case, an initial survey “yes” response helps to set up the later

opportunity to free ride with respect to the actual contribution. For the private goods case, a

“yes” response (would purchase) to the survey encourages the production of the good while the

agent gets to decide later whether to purchase the good. Thus, if the agent anticipates any

positive probability of wanting to purchase the good, then a “yes” response is optimal. If the

agent anticipates that the good will be offered irrespective of the responses given by agents but

the agent perceives that the responses may influence the price of the good, then it is optimal for

the agent to appear more price sensitive than is actually the case. This result is often seen in

marketing research where agents have been found to more price elastic in surveys than in actual

market purchases. The only problem with these cases from the perspective of economic theory is

not whether there should be a divergence between actual behavior and the survey estimate, but

rather, whether the magnitude of the divergences empirically observed should be even larger.

There are other interesting implications of the lack of incentive compatibility for binary

discrete choice survey questions for voluntary contributions and the introduction of new private

goods with respect to other anomalies such as insensitivity to the scope of the goods being

valued. For instance, as long as the good is potentially desirable it is optimal to say “yes” to the

survey question. The scope of the good and its cost do not influence this decision unless the good

becomes so small that even if at a zero cost it is not desired or if the cost becomes so high that it
16

would never be purchased. In both of these later instances, either a “yes” or a “no” response by

the agent will have the same effect on the agent’s utility.

If the binary choice is between two different forms of a quasi-public or private good, then

desirable incentive properties can be restored as long as only potential users are interviewed.13

To see this, consider the classic case of a campsite. At present the campsite is unimproved and

has a low entrance fee (possibly $0). The alternative is to improve the campsite and increase the

entrance fee. The agent should now choose the status quo campsite price/quality combination or

the alternative campsite price/quantity combination to maximize utility. This binary choice can

be shown to have identical properties to the RARAS survey mechanism. The property that the

mechanism needs to be incentive compatible is the ability of the agency to force one of the

alternatives on a particular agent irrespective of that agent’s preferences in a situation where the

agent’s utility is influenced by the agency decision. Two important caveats should be kept in

mind. First, in this situation the total number of times the good will be used under the alternative

is endogenous. In our campsite example, if the higher quality-price campsite alternative provides

more utility than the status quo, the anticipated number of visits to that campsite under that

alternative may be larger or smaller than under the status quo. Second, for agents whose

probability of use of the good does not differ between the two configurations, any response has

the same impact on the agent’s utility. This problem is not usually seen because most

recreational surveys are either done on site or from lists of users. Marketing researchers typically

screen out non-users of a product class before asking preference questions.14 The risk in both

 

13 Quasi-public goods are those provided by the government but for which it is possible to exclude members of the

public from using. This exclusion can occur in terms of charging a price to use the resource, having the agent spend

money or time to use the resource or by having the resource effectively bundled as an attribute of a privately

purchased product. Common examples include government campgrounds and houses located on public lakes.

14 There are exceptions. Boxall, et al. (1996), for instance, ask hunters in Alberta about two different

management/cost regimes for a specific area that few currently hunted in and few were likely to hunt in with the
17

instances is that focusing on current users of the good will miss those who would likely use the

good if its quality/price attributes were changed.

This choice between two configurations of a good works for public goods and private

goods too, irrespective of the nature of the payment obligation, as long as the agent desires the

good at no cost. To see this, consider a private charity that wanted to build one of two different

monuments in the center of town. The charity conducts a survey of the public to determine which

monument is preferred and the higher the level of support for a particular monument the more

likely that monument will be built. The agent should pick the preferred monument since this

increases the agent’s utility more than the alternative monument and neither monument imposes

any cost on the agent. Our favorite example of a private good question is the bar owner that

surveys patrons and asks whether they would prefer to have the bar’s sole draft beer, currently a

domestic brand priced at $1, switched to an imported brand at $2. The bar patron should pick the

import only if having that alternative available provides more utility than the domestic. Note that

the number of beers that will be purchased is not revealed by the agent’s choice and could go up

or down.

Table 1 summarizes the incentive properties of binary discrete choice questions by the

type of good and the payment characteristics. In this case we have assumed that the agent would

desire the good if there was no cost, otherwise the incentive properties of the question are

undefined. What is striking is that anomalies with respect to a divergence between estimates

based on stated preferences and estimates based on behavior are heavily concentrated in the two

cases that are not incentive compatible.

 

alternative management scheme. In this instance, the contingent valuation estimate was dramatically larger than the

travel cost estimate, something that is fairly unusual in comparisons between the two approaches for quasi-public

goods (Carson et al., 1996). When the estimate of the change in the probability of use is used to scale the CV

estimate, the two approaches result in quite similar estimates.
18

Table 1: Incentive Properties of Binary Discrete Choice Questions

Type of Good Incentive Property

New public good with coercive payment Incentive compatible

New public good with voluntary payment Not incentive compatible

Introduction of new private or quasipublic good

Not incentive compatible

Choice between which of two new public

goods to provide

Incentive compatible

Change in an existing private or quasipublic goodIncentive compatible but choice does not

reveal information about quantities

The second key assumption in the discussion of the RARAS mechanism is that is that of

a single up-down vote on a single issue. It is also not possible to relax this condition and there

are several common instances where it is violated. The best-known ones are the rules for school

bond referendums in many areas (Romer and Rosenthal, 1978; Lankford 1985). The school

board gets to propose the level of educational inputs and the tax rate. However, if the referendum

is voted down, the school board can only bring up another referendum measure with a level of

educational inputs and a tax rate that is lower than those voted down but higher than the default

status quo. A respondent who prefers the initially offered bundle to the status quo may

nonetheless have an incentive to vote against it in order to gain opportunity to vote in favor of an

even more preferred provision/tax package. With respect to valuation of an environmental

project, Richer (1995) shows that his CV WTP estimates are influenced by information about

whether a different alternative plan for a national park in California’s Mojave Desert was likely

to be put forth if the current plan described in the survey was not approved. Another variant is

where there is another party (e.g., another government agency or private entity) who potentially
19

can provide the good.15 The general principle is that direct linkage between a decision on one

issue and a decision on another issue can cause difficulty in interpreting the result, as the optimal

response of the agent should generally take the sequence of decisions and options into account.

There is a further condition that is important for the interpretation of the results but not

for the incentive properties of the RARAS mechanism. The agent needs to believe that if the

agency implements a particular alternative: the specified good, Q, will be provided and the stated

price, P, assessed. If instead the agent believes that Q* will be delivered and P* paid if this

alternative is chosen by the agency, then the agent’s optimal response should be based upon (Q*,

P*) not the stated (Q, P). Note this condition holds for interpreting actual votes or actual

consumer purchases as well as responses to preference survey questions.16 An important

implication of this condition though is if the goods and prices used in a preference survey go

 

15 This problem appears to have influenced the Cummings et al. (1997) results. In that experiment, agents are

randomly assigned to a “hypothetical” treatment and a “real” treatment in which the group votes on whether to

contribute a specified amount per agent to provide the good. The estimate based upon the hypothetical treatment is

higher than that of the real treatment, although Haab, Huang, and Whitehead (1999) show judgment of the

significance of the difference depends upon how the larger variance in the “hypothetical” treatment is taken into

account. We believe that to many of the agents interviewed in Georgia, the Cummings et al. hypothetical treatment

should have appeared as an attempt to determine whether it was possible to mount a fundraising effort to provide

printed information booklets on toxic hazards to poor people in New Mexico. As such, we would have expected the

“hypothetical” treatment WTP to be higher than true WTP. However, uncertainty about why agents in Georgia

should be asked about voluntary contributions to a New Mexico program may have lead to the larger variance found

by Haab, Huang, and Whitehead (forthcoming). For the “real” treatment we would have expected an under-estimate

of true WTP due to the possibility of having some other group pay to distribute the already printed booklets. A later

experiment by Cummings and Osborne-Taylor (1998) effectively replicates this experiment but with additional

treatments where there are different probabilities that the vote taken by the group is binding. The WTP estimate

decreases from the “hypothetical” treatment to the “real” treatment as the probability that the group vote is binding

goes from 0 to 1. This is the result that our model predicts if all treatments were perceived by agents as being

consequential and that there are competing incentives to over pledge and free ride in all of the probabilistic

treatments. The result that would be predicted theoretically if there was no incentive to over pledge in the

“hypothetical” treatment and free ride in the “real” treatment would be that all of the treatments with a positive

probability of the vote being binding should result in similar WTP estimates.

16 Carson et al. (1994) show, for instance, in a recent CV study in California that respondents who do not currently

pay taxes are willing to pay more than respondents with otherwise identical characteristics. That respondents who

believe that the state government would assess the one time tax in multiple years are willing to pay less than

respondents who think the fee will only be applied one time and that respondents who don’t think that the plan will

work completely are willing to pay less than those who think that it will work. See Randall (1994) for a discussion

of this issue in the context of the travel cost model. There are large literatures in marketing and political science

dealing with what are effectively the P’s and Q’s perceived by agents when they make decisions.
20

beyond what the agent finds plausible, the preference survey question is likely to be answered on

the basis of the expected good and the expected price rather than the stated ones.

Introduction of Cost Uncertainty

Binary discrete choice preference surveys often provide a cost (in monetary or other

terms) for each alternative and this cost information plays a key role in estimating welfare

measures. What role should agent uncertainty over cost play in the answers given? The answer is

obvious if the survey provides a cost estimate of $X and the agent thinks that since the

government has a proclivity for cost overruns that the actual cost will be double the stated cost.

The analysis should be performed with the cost as perceived by the agent.

The more interesting case is when the agent takes the survey and provides $X as the

expected value with some type of distribution around $X. Here the key issues can be seen to

revolve around whether the original status quo choice set will still be available and whether a

commitment to the pay for the good is required ex ante before the cost uncertainty is resolved.

These two conditions determine whether shifts from an original “yes” to a “no” and vice versa

are possible given a mean preserving increase in cost uncertainty. Table 2 displays the possible

outcomes.

Table 2: Effect of Increased Cost Uncertainty upon Binary Choice

Ex ante choice

(i.e., commitment)

Ex post choice

(i.e., no commitment)

Status Quo

still available Can only shift Yes ® No Can only shift No ® Yes

Status Quo

no long

available

Can only shift Yes ® No

Can shift either

Yes ® No or No ® Yes
 For the case of provision of a public good with a coercive payment mechanism, the

status quo choice set is still available but one has to commit ex ante to paying the uncertain cost.

This commitment translates into income uncertainty and hence is never preferred by risk adverse

agents. Hence one would expect to see some shifts from “yes” to “no” responses. There should

be no shifts in the opposite direction, so that the aggregate change is a decline in standard

statistics of the WTP distribution like the mean and median relative to the case with no cost

uncertainty. The other case where an ex ante commitment is required has the same result but may

be of less practical relevance since most examples here require an ex ante commitment to

purchase a fixed quantity of the alternative to the status quo before the actual cost of the

alternative was observed.

 The opposite phenomena, possible shifts from original an “no” to “yes” response with

increases in cost uncertainty, should occur in the case where the choice can be made ex post after

the cost is observed and the status quo choice set is always still available. The main examples of

this case are provision of a pubic good via voluntary contributions and the introduction of a new

private good. The basic logic in this case is that since the status quo choice set will still be

available, all agents will either favor or be indifferent the addition of the new alternative.

Increasing the level uncertainty can cause some agents who were indifferent to the addition of

the alternative to the choice set to favor it. Changes the a “yes” to a “no” response cannot occur,

even though it is possible that an increase in cost uncertainty can make some agents, who were

already in favor, worse off.

The last of the four cases occurs where the only ex post commitment is required and the

original status quo choice set will no longer exist, if the alternative to the status quo is provided.

The main examples here are quasi-public goods and private goods where only one of two
22

possible configurations of the good will be offered (e.g., a low quality-low price recreation site

could be transformed into is a high quality-higher priced version of the site) In this case, it

obviously possible for increasing the degree of cost uncertainty to result in both shifts from “yes”

to “no” and “no” to “yes.”

There are a number of other informational issues that we do not explore here except to

note that a formal analysis of the role of different types of uncertainty is likely to be more

productive than the all too frequently invoked vague concept of agent unfamiliarity with a good

as a justification for all types of apparent aberrant behavior. Much of the richness of economic

theory in recent years has come from the introduction of different types of uncertainty and asking

how agents should optimize in the face of it (Varian, 1992). Particularly, relevant here is the

rapidly developing literature on how agents process information in candidate and referendum

elections (e.g., Popkins, 1991; Lupia, 1994). This literature suggests ways in which agents make

reasonably informed decisions based on imperfect information. Further, simply providing more

information does not necessarily lead agents to make the decisions closer to that which they

would make if fully informed (Lohmann, 1994).17 This suggests that the informational content of

a survey used for environmental valuation should be examined to see if agents were given a

reasonably complete, comprehensible, and balance presentation of the alternatives offered.

 

17 For example, consider an agent who initially favored a project and saw both its benefits and costs as being small.

The agent, if fully informed, would still favor the project but realized that both its benefits and its costs were large.

The agent informed that the cost of the project is large, but not given the corresponding benefit information, will

now oppose the project. Much advertising in marketing and political campaigns operates on this notion of providing

selective “half-truths.”
23

Double-bounded Discrete Choice Questions

The inherent problem with a binary discrete choice question is the limited information the

response to it provides about the agent’s preferences.18 Double-bounded discrete choice

estimators have become popular in the environmental valuation literature because they tend to

dramatically shrink the confidence intervals around point estimates of statistics of the willingness

to pay distribution. The approach is straightforward. If the agent said “yes” to the initial cost

amount asked, then ask the same question at a pre-chosen higher amount, and if the agent said

“no” to the initial amount, ask the same question at a lower amount.19 The initial presentations of

the double-bounded format relied on double sampling/interval censoring statistical models

(Carson, 1985; Carson and Steinberg, 1990; Hanemann, Loomis, and Kanninen, 1991). They

assumed that agents have a single latent WTP value and that the responses to both the first and

the second questions are based upon simply comparing this latent WTP value to the cost amount

asked about in each question. Statistically, the implication of this assumption is that, with

appropriate conditioning, there is perfect correlation between the WTP distributions implied by

the responses to the two questions.

 

18 The only information provided is whether the agent’s WTP for the good is higher or lower than the single amount

asked about in the survey question. It is possible to use parametric assumptions about the underlying WTP

distribution to effectively overcome this sparse information, but such assumptions can play a large role in the

estimates derived. Non-parametric approaches to the use of binary discrete choice data (e.g., Kriström, 1990) exist

that make the power of these assumptions abundantly clear.

19 In some respects, the double bound model is similar to the iterative bidding game approach used in the early CV

literature (Randall, Ives, and Eastman, 1974) that was usually found to suffer from a phenomena known as starting

point bias whereby the amount initially provided the agent influences the agent’s final WTP amount. There are some

key differences though which make the two approaches fundamentally different. The initial cost amount in the

iterative bidding game was never intended to reveal information about the goods actual cost and the iterative steps

from that amount are usually quite small. In contrast, the statistical tools used to analyze data from both the binary

discrete choice and the double bound discrete choice formats exploit the agent’s conditioning on the cost number

explicitly provided and the interval formed by the first and second price is fairly large. Most good studies using a

double-bounded format go to some effort to provide a rationale to the agent as to why the cost number used in the

second question is different from that of the first. An interesting variation on the double-bounded format is a single

binary discrete choice format with a follow-up open-ended question. Farmer and Randall (1996) analyze this format

from a theoretical and empirical perspective and show results similar to those described here for the double-doublebounded estimator: the second responses tend to be biased downward.
24

Following Cameron and Quiggan’s (1994) pioneering examination of this assumption,

several stylized facts have emerged concerning the comparison of the WTP estimates based on

the first binary discrete choice question and both binary discrete questions: (a) the WTP

distributions implied by the first and second questions are not perfectly correlated, (b) the WTP

estimate based upon just the first estimate is higher than the WTP estimate based upon both

questions, and (c) the number of negative responses to the second question is higher than would

be expected based upon the WTP distribution estimate from the first question alone. Alberini,

Kanninen, and Carson (1997) have put forth a general error-components model, and McLeod and

Bergland (1999) have put forth a Bayesian preference-updating model to handle these issues.

What sort of effects should the asking of a second binary discrete choice question have

on the latent WTP distribution? The key property of this format from our perspective is that the

agent has been told that the same Q was available at two different prices. The best-case scenario

here is that the agent takes the second price as the expected price but now considers the price to

have some uncertainty surrounding it.20 Consistent with the discussion in the previous section,

statistics such as mean or median WTP will be shifted downward in the second question for risk

adverse agents and public goods even though preferences for it have not changed.

There are, however, several other plausible alternatives for what the act of asking the

second price should signal to agents. One of these is that the agency is willing, in some sense, to

bargain over the price. For agents who originally answered “no” and got asked a lower price, the

optimal response may be to answer “no” again in hopes of getting offered an even lower price.21

This should result in the second WTP response being “no” for some of these agents even though

 

20 Alternatively, if the agent thought the first price had some uncertainty surrounding it, asking the second price

should increase the original level of uncertainty since for the double-bounded estimator the first and second prices

are typically fairly far apart.
25

had this amount been asked at the first question the response would have been “yes.” A similar

effect can be found with respect to those whose original answer was “yes”. Since the good was

originally offered at a lower price, it can presumably be provided with some positive probability

at the initial price. As such, some agents will find it in their self-interest to risk not getting the

good by holding to the lower price and saying “no” to the second higher price, even though the

agent’s WTP for the good exceeds the second price. The effect of this type of behavior would be

to shift the WTP distribution implied by the second question to the left, and hence, reduce

estimates of mean and median WTP.

Another plausible assumption is that the actual cost to an agent will be some type of

weighted average between the two prices. If this assumption is made, the second question should

be answered on the basis of this weighted average of the two prices. It is straightforward to see

that for an initial “no” response, that any weighted average of the first and second prices is

higher than the second price. For an initial “yes” response, any weighted average of the first and

second prices is lower than the second price.22

The last plausible assumption we consider is that the agent might interpret the signal

given by the second price as implying that the quantity has changed to match the changed price

in a consistent manner. For an initial “no” response, the shift in quantity that is consistent with

the reduction in price is to reduce the perceived quantity/quality of the good that would be

provided. The implication of this is to shift the WTP distribution implied by the second response

to the right for these respondents. This is a commonly voiced concern in focus groups and

debriefing questions. For agents who initially said “yes”, the shift in perceived quantity is

 

21 It is of course possible to expand the double-bounded concept to asking a third question. See Bateman et al.

(1995) for an example.
26

upward. There does not appear to be any collaborating evidence to support the proposition that

this is a common phenomenon.

What should be grasped from this discussion is that to a rational agent the second price

must signal that something is going on. All of the plausible assumptions lead to the correlation

between the WTP distributions implied by the two questions being less than 1. All of these

assumptions also shift the WTP distribution implied by the second question to the right for

agents who initially gave a “no” response, and hence, produce an “excess” number of no-no

responses. For agents initially giving a “yes” response, it is possible for the WTP distribution

implied by the second question to shift either to the left or the right. However, only the price

averaging assumption has much credence in terms of the possibility of producing an upward shift

in the standard WTP statistics. On balance, we would expect that WTP estimates from a doublebounded format to be smaller than those from a single-bounded format. All of these hypotheses

tend to be strongly supported by the empirical evidence. It may be desirable to use the doublebounded format in CV studies; however, this desirability rests on the analyst’s tradeoff between

the likely downward bias and the tighter confidence interval (Alberini, 1995).

Continuous Response Formats

Ideally one would like to have the agent’s actual WTP or WTA, not a discrete indicator

of it. So it is not surprising that many early CV studies used an open-ended direct question. 23

Many economists thought that these early efforts would fail because agents would give the

 

22 Note that this assumption is not inconsistent with the arguments concerning uncertainty and the two may be

combined. For initial “no” responses, this effect of adding uncertainty is reinforcing in a downward direction. For

initial “yes” responses, the effect is in the opposite direction and mitigates the upward effect of price averaging.

23 The continuous response format is known as a matching question in the psychology literature and is a special type

of open-ended question in the survey research literature.
27

extremely high WTP answers. This did not happen (e.g., Brookshire, Ives, and Schulze, 1976),

and interest in the survey based valuation methods grew in part due to this anomaly.

The early problem that researchers did find with the direct question was that agents

always wanted to know what the project would cost them. Agents did not understand why they

were not provided the cost information if the agency had worked out the details of how the good

would be provided. Further, many agents appeared to have great difficulty formulating a

(continuous) WTP response. This led to very high non-response rates and a large number of socalled “protest zeros” which were typically dropped from the analysis. This lead to speculation

that survey respondents did not have “well-defined” preferences in an economic sense.

Three different directions were tried to overcome this problem. The binary discrete

choice format (Bishop and Heberlein, 1979) discussed earlier gets around one of the key

problems by giving agents the cost number they want and then uses a statistical analysis that

“appropriately” conditions on agents reacting (favor/not favor) to that cost number. The earlier

iterative bidding game method suggested an initial amount and iterates up or down from that

amount in small increments (Randall, Ives, and Eastman, 1974). The payment card approach

asks agents to pick a number (or any number in between) on a card (Mitchell and Carson, 1986;

Cameron and Huppert, 1991). The latter two methods can come close to achieving a WTP

response in continuous terms; and, except when these formats have special properties, the

discussion of the continuous response format will apply to these formats as well.

With different elicitation formats came the inevitable urge to compare their results (e.g.,

Smith and Desvousges, 1986). Researchers were dismayed to find that different response formats

lead to different WTP estimates and the divergence between these estimates is frequently cited as

one of the major reasons why estimates based on stated preference questions should be rejected
28

(Hausman, 1993; McFadden, 1994).24 The stylized fact here is that discrete choice formats

produce higher WTP estimates than do continuous response formats (e.g., Boyle et al., 1996).

Should the divergence in estimates from different formats be surprising?25 No. Given the

Gibbard-Satterthwaite result, it is impossible to formulate a continuous response question that

has the same incentive and informational properties as an incentive-compatible binary discretechoice question. Many researchers looking at the results, however, have been misled by the facevalue dilemma. The divergence between the estimates from the different formats suggested that

either agents were not truthfully revealing their preferences to one or more of the elicitation

formats or that they did not have well-defined preferences in the sense used by economists.

As noted earlier in this discussion, the expectation of many economists was that most

agents would provide very large WTP responses when asked an open-ended WTP question if

agents were acting strategically but not truthfully revealing their preferences. However, the

opposite phenomenon was observed: estimates from binary discrete-choice questions were

higher than those from continuous response CV questions and continuous response CV questions

contained lots of zero responses.

Faced with an open-ended question, a very large WTP response does turn out to be the

optimal strategy for an agent who believes (a) the cost of the public good to the agent is fixed,

(b) her true willingness to pay for the good is larger than the cost if provided, and (c) the good is

more likely to be supplied the larger the sum of the willingness to pay responses given by agents.

 

24 The irony in this position is that estimates of other economic quantities based upon substantially different

econometric techniques have typically differed even though data on actual behavior was being used. The usually

recommended approach in this situation has not been to discard economic theory and econometric methods but

rather to understand the source of the differences.

25 From the critique by cognitive psychologists, the divergence between framing provided by the (binary discrete)

choice and open-ended matching question is at the heart of problems with microeconomic theory (Tversky, Slovic,

and Kahneman, 1990).
29

Note that only the subset of agents whose WTP is greater than their cost should be giving a

positive WTP response, so one should never have expected all agents to engage in this behavior.

Condition (c) corresponds to the benefit-cost criteria, but it is hard to find a single

instance where an agency decision has been made based purely on that criteria. There is little

evidence to suggest that agents believe that the agency is simply summing their WTP responses.

As such, we believe it useful to consider a variety of other beliefs that agents may hold.

Let’s first consider the optimal response of an agent whose perceived cost of the public

good is greater than the agent’s willingness to pay. Maintaining the previous assumptions, this

agent’s optimal response is “zero”. This result turns out to be fairly robust to plausible

alternatives to (c) that we discuss below and, as such, may help to explain the large number of

zero responses received to open-ended type questions. The intuition behind this result is that the

agent’s utility is reduced if the public good is provided and the cost assessed against the agent.

The response that adds the least amount to the sum of the benefits (given the usual nonnegativity constraint in the open-ended format) is “zero.”

Step back for a moment from the benefit-cost criteria that has dominated economic

thinking on the incentive structure of the open-ended question and recognize that the simple act

of asking an open-ended question is likely to signal to agents that the cost allocation among

agents for providing the good is not fixed. Once the agency is prepared to shift the vector of

costs facing agents, changing condition (a) above, toward increasing the cost to agents having

(relatively) high WTP for the good and decreasing it to those who do not, the incentives for

agents whose WTP is greater than the initially perceived cost change substantially. These agents

now have to balance the increased probability that the good will be supplied with a high WTP
30

response against the potential upward shift in the cost they will pay if the good is provided. For

agents having WTP less than the initially perceived cost, the optimal response is still zero.

Since the government rarely if ever uses a pure benefit-cost criteria, it may be plausible

for agents to assume that the agency is simply trying to determine what percentage of the

relevant population has a WTP higher than the cost which may or may not be assumed to be

known to the agency at the time of the survey. Combined with the potential to reallocate the cost

burden, the optimal response of an agent whose WTP is greater than the initially perceived cost

is now equal to the cost while the optimal response of an agent whose WTP is less than the

initially perceived cost the optimal response is still zero.

In all of these cases, the optimal response depends strongly on the agent’s perception of

the agency’s cost of providing the good. The agent should first compare her actual WTP to the

expected cost. The optimal response for agents whose WTP is less than the perceived cost, under

most plausible uses of the information provide, is zero. Such an agent should additionally

“protest” in any other way possible, as the change from the status quo will negatively impact the

agent’s utility. The optimal response for an agent whose actual WTP is greater than expected

cost depends upon her belief about how the agency will use the stated WTP. In this case, the

optimal response typically is further conditioned on expected cost. The difficulty in interpreting

the positive WTP response is that different agents may have different beliefs about agency use of

the response.

Agents, however, don’t know the cost with certainty. They can formulate priors about the

cost and should incorporate any information provided in the survey that they believe is related to

cost. This type of behavior would give rise to starting point bias in iterative bidding games

(Boyle, Bishop, and Welsh, 1985) and range/placement effects in studies using payments cards
31

(Rowe, Schulze, and Breffle, 1996) to the extent that agents think that the “extra” information

provided in these formats is correlated with cost.

On occasion, a variety of different open-ended formats that are said to be incentive

compatible are used in a survey context such as the Becker-DeGroot-Marshack mechanism or

the Vickery auction. 26 Both of these mechanisms elicit a continuous WTP response. There are

two things to remember about such mechanisms. First, they do not get around the GibbardSatterthwaite result. Holt (1986) and Karni and Safra (1987) (hereafter HKS) independently

showed such mechanisms depend crucially on the preferences obeying the expected utility

assumption. Many researchers are willing to maintain the expected utility assumption, and many

key economic results on risk are locally robust to most non-expected utility alternatives

(Machina, 1995). However, when trying to implement either of these two mechanisms in a

survey context, the difficulty lies much deeper. Both of these mechanisms rely on the ability to

condition the agent’s response on an “exogenous” random element. We have shown that it is

impossible to formulate a simple open-ended matching question that is informationally and

strategically equivalent to an incentive compatible binary discrete choice question in a survey

context. This result is a companion of the HKS theorem. To make the matching question

equivalent strategically to the binary discrete choice, the agency has to pre-commit either to the

cost or to an exogenous device to provide the cost. Doing so prevents the agency from exploiting

the extra information that the agent provides in the matching format but not in the choice format.

To get the agent to reveal the matching answer, the agent cannot know the cost. The need for the

agent’s uncertainty about the cost puts one back in the HKS world where expected utility is

 

26 Other mechanisms eliciting a continuous response like the Groves mechanism (Groves, 1973) require stronger

restrictions on preferences (e.g., quasi-linearity in income) and the possibility of side payments.
32

required. The need for agency pre-commitment not to exploit the extra information contained in

the continuous WTP response effectively prevents its being used in a survey context.

Sequence of Paired Comparisons

In addition to wanting tighter confidence intervals on the WTP distribution for a single

good, decision makers often want information on the WTP distributions for a variety of related

but different goods so that they can pick the best option. There are two popular approaches in the

literature for doing this. The first is to offer agents a sequence of paired comparisons. The second

is to ask agents to pick between or rank order a set of k > 2 alternatives. Here we discuss the

strategic issues that arise with these choice formats and do not deal with issues related to the

adequacy of the information set on each distinct good provided in the survey.

In an ideal world in which the objective involves valuing public goods, the agent treats

each paired comparison independently and the desirable properties of a single binary discrete

choice question with a coercive payment requirement can be repeatedly invoked. There is a very

simple question, however, that illustrates the fundamental difficulty with a sequence of paired

comparisons. Consider the case of air pollution levels in a city. The agent is asked to pick

between different pairs of air pollution levels that involve different costs and different health

effects and visibility levels. Since air pollution in the city is a public good, however, all agents

will eventually face the same air pollution level. If k different air pollution levels are described to

the agent in the course of the sequence of paired comparison, the agency must have some method

of choosing among the k different levels. Any particular method that the agent perceives that the

agency is using to incorporate agent preferences into its choice of an air pollution level generally

will provides incentive for non-truthful preference revelation. In some instances, it will even be
33

optimal for the agent to reject his or her most preferred level (out of the k) in a particular paired

comparison. Once this is possible, the standard methods of inferring value from choices no

longer work. The essential problem is that an agent’s optimal choice depends both upon the

agent’s preferences, expectations about what the other agents will do, and the perceived rule for

aggregating the results of each paired comparison. This result has long been established in the

literature on the properties of voting rules (Moulin, 1994).

With quasi-public and private goods, the difficulties noted for public goods still exist,

with the exception that it may be possible for more than one of the k goods to be provided. This

possibility tends to reduce the likelihood that an agent will make a choice that is not her favorite;

and in the next section on multinomial choice, we discuss the aggregation issue further.27

Multinomial Choice Questions

Many of the issues raised in the previous section on a sequence of paired comparisons are

relevant to multinomial choice questions. The strategic issue that an agent faces when answering

a multinomial choice question (pick the most preferred out of k > 2 alternatives) is how the

agency translates the responses into actions. The simplest case consists of generalizing the

decision rule used in the binary discrete choice format by assuming that the agency will provide

only one of the k goods and that the higher the percentage of the sample picking any particular

alternative, the more likely that alternative will be provided. The well-known result from the

 

27 There are further issues related to a sequence of paired comparisons that need to be addressed in any particular

analysis. The first of these is the strong possibility that the scale term associated with each paired comparison is

different. If this is the case then much of the gain in precision and the ability to deal with changes in attributes

associated with asking the sequence of paired comparisons may be an illusion. The second is that most rules for

combining information from different paired comparisons implicitly require that the independence of irrelevant

alternatives assumption to hold. This property is routinely rejected in paired comparison data. The third involves the

common use of pairs where both alternatives are off the agent’s current utility frontier and neither represents the

status quo. This practice requires much stronger assumptions about the nature of the agent’s utility function than

typically assumed in order to combine the data from different paired comparisons.
34

voting literature on multi-candidate races with a simple plurality winner is that the race from an

agent’s strategic perspective reduces to a binary choice between the two alternatives that the

agent believes will receive the largest votes independent of the agent’s vote. The rationale behind

this result is straightforward: only the top two alternatives have a chance of winning; picking the

most preferred alternative among these two will maximize the utility of the agent’s final

outcome.28 The agent is truthfully revealing her preferences, but such truthful preference

revelation is, as it should be, conditional on the expectations about the choices of the other

agents. However, the agent is not answering the question of interest to the analyst. It will be

optimal in many instances for the agent to pick an alternative other than the (unconditionally)

most preferred one.

Let us now consider perhaps the opposite case, one of particular relevance to private and

quasi-public goods, by changing one of the key assumptions. Now instead of only one of the k

goods being supplied, let k-1 of the goods be supplied. To keep matters simple, assume further

that the agent only uses at most one of the goods. Examples of such a choice context might be a

government agency that had to close four out of five recreational fishing lakes or a computer

company that was going to offer four out of five configurations of a particular computer model.

In this case, it is optimal for the agent to pick the most preferred alternative out of those offered.

Formally, it can be shown that this case collapses to a binary discrete choice of the agent’s most

preferred alternative against another stochastically chosen alternative. To see this, note that the

worst possible outcome for the respondent is that the agent’s first choice is not made available.

 

28 With a richer model of agent expectations, it may be optimal for the agent to vote for an alternative that is not one

of the top two if there is enough uncertainty over the expected finish of alternatives and the utility differences

between the alternatives is large enough. The manifestation of this proposition can be seen in the behavior of fringe

political candidates in plurality winner elections. Such candidates try to convince voters that they have a non-trivial

chance of winning, that the difference in positions between the two front-runners is extremely small, and that they

are much closer to the voter’s ideal point.
35

Because all of the other alternatives are provided, the agent’s second choice will be available.

Effectively, this is a determination of what alternative will not be provided and in pairing the

agent’s favorite alternative against any of the other alternatives, the agent’s optimal response is

to pick her most preferred.

The general result we have shown is that, if all but j of the alternatives are to be provided,

then the alternative chosen by the agent should be one of the agent’s j favorites. Often the

number of alternatives that will be provided is unknown to the agent at the time of making the

multinomial choice. A stochastic version of this result has the agent trading off the utility of sets

of alternatives with different maximum elements against the agent’s prior on j and the agent’s

priors on the choices made by the other agents. Doing so reveals that agents will pick either their

(unconditionally without considering the responses of other agents) favorite alternative or close

to it, as long as one of three conditions holds: the expectation of j is fairly small, the utility

difference between the agent’s most favorite alternatives and the other alternatives is large, or the

prior on the choices by the other agents is fairly uninformative. The implication of this is that

agents will appear to make mistakes or optimization errors more often. If they don’t pick their

favorite, they should pick an alternative close to it.

The statistical manifestation of this type of behavior is a violation of the error term

properties associated with the independence of irrelevant alternatives (IIA) assumption. In

empirical applications of this elicitation format, the IIA assumption is usually violated. While

there are a number of other good reasons for this assumption being violated, such as the rationale

behind classic red bus-blue bus problem, it is usually impossible to separately identify the reason

for an IIA violation. From a purely statistical viewpoint, it is possible to deal with the problem

by introducing one or more scale/variance terms (Swait and Louviere, 1993). This is often
36

sufficient for looking at marginal tradeoffs between attributes. To uniquely recover the latent

WTP distribution, it is necessary to have an estimate of the correct scale factor.29 The optimal

strategic behavior in this case is often observationally equivalent to direct manipulation of the

scale parameter and making recovery of the correct scale factor impossible.30 Tests for whether

data from stated preference surveys and revealed preference observations are consistent with

each other and can be combined after (potentially) allowing for a difference in the scale factor

(Adamowicz, Louviere and Williams, 1994) are tests against random responses in the stated

preference data, not tests against strategic behavior.

With either subadditivity or superadditivity of the utility of the different alternatives and

k-j (j > 1) alternatives to be provided, it is possible to find conditions where the agent should

indicate her unconditionally least preferred alternative. The rationale here is that the agent’s

outcome utility is defined on the set of goods to be provided, not the individual goods taken

independently. This is a hopeless situation for learning anything reliable about agent preferences

for individual goods.

An alternative to asking agents to pick their single most preferred alternative out of k, is

to ask them to rank order all k alternatives. This exercise could potentially provide considerably

more information, but an analysis of the agent’s strategic incentives becomes considerably more

difficult. The same issue for the agent still exists: how does the agency translate the ranks into a

choice of which of the k alternatives to provide. Methods for dealing with rank data in a manner

 

29 This correct scale factor can often be obtained in studies involving quasi-public or private goods from a model

estimated on the more limited set of choices current available in the market. This will not typically be the case for

public goods.

30 The scale parameter is typically the negative inverse of the price coefficient. The agent’s optimal strategy is to

induce the agency to supply the good with the most desired set of attributes at the lowest price. The simple way to

do this is to pick the favorite anytime the price is low and otherwise pick something close to it with a low price.

Formulated in terms of the expected minimum cost that the agent believes the agency would provide the good at, the

agent wants to appear to have an infinite demand elasticity at this cost and to be uninterested above that cost.
37

consistent with economic theory effectively require the IIA assumption to hold for all possible

subsets of the ranked data. This implies that it is possible to explode the data to form sets of

multinomial choice questions (Chapman and Staelin, 1982). The IIA assumption can be tested

but it does not appear to generally hold for contingent ranking data, and welfare estimates can be

substantially impacted if the IIA assumption does not hold (e.g., Hausman and Ruud, 1987).31

Concluding Remarks

We have argued that serious consideration should be paid to the incentive and

informational properties of preference questions. Much of the difficulty with interpreting the

apparent anomalies32 associated with the estimates based on preference survey questions

revolves around what we call the face-value dilemma: either agents always truthfully reveal their

preferences to survey question as stated or they do not. This is a false dilemma.

Simple common sense economic models predict large divergences between what agents

say they will voluntarily contribute to provide a public good and what they actually contribute.

There are now many studies that demonstrate this prediction empirically. The difficulty lies not

in the theory or the experimental demonstration but rather in the interpretation that is often

placed on these results. Rather than be taken as evidence that respondents don’t have well-

 

31 A major problem occurs when there are a group of respondents who do not appear to want to trade-off one of the

attributes against money. The appearance of such lexicographic preferences can lead to infinite WTP estimates. A

subtler problem occurs in that the variance of the error term appears to be substantially larger for “middle” ranks

than the most and least preferred alternatives.

32 The term anomaly is often loosely used. It is possible to have results that represent anomalous behavior from the

perspective of economic theory and it is also possible to have such behavior occur in a survey. The most interesting

anomalies from the perspective of this paper are those that only occur in surveys. The first step to take with such an

anomaly is to see if it can be observed in settings not involving surveys. A number of anomalies first alleged to be

survey specific have been shown to be easily replicable in experimental contexts and examples readily identifiable

in common market transactions. These include preference reversals (Grether and Plott, 1979), large divergences

between WTP and WTA (Bishop and Heberlein, 1990), and part-whole bias (Bateman et al., 1997). In some of these
38

defined preferences, differences between the estimates obtained using different elicitation

formats, if predicted by economic theory, should be taken as evidence supporting the proposition

that respondents are taking the scenario posed seriously.

Divergences between binary discrete choice and double-bounded formats or between

binary discrete choice and open-ended formats are likewise consistent with theory. Optimal

response strategies in most cases are fairly simple, and in many instances, such as the zero

responses to open-ended type questions, fairly robust to alternative assumptions made about

agent beliefs. In some situations, particular elicitation formats should be avoided altogether,

while in others one faces a classic bias versus variance trade-off. The researcher should

understand the trade-off being made in the choice of an elicitation format.

Claims about the specific incentive and informational properties of a particular elicitation

format should not be made in the abstract. Careful attention needs be paid to the type of good

being offered, the nature of the payment obligation for the good, and other aspects of the context

in which the good is offered in order to clearly determine incentive and informational properties.

For the binary discrete choice format, the introduction of a new private good turns out to be one

of the worst cases for truthful preference revelation. The other bad case is to compare survey

indications of willingness to voluntary contribute to provide a public good to actual

contributions. Here neither estimate should approximate the true underlying WTP. One need not

cast a binary discrete choice question explicitly as a vote in a referendum to get an incentive

compatible question; it is sufficient to structure the question as advice to the government on the

issue, a result that should be of use to researchers in areas where referenda are not frequently

held.

 

instances, such as the often-noted WTP-WTA divergence, models predicting such divergences consistent with

standard neoclassical economic theory have been proposed (Hanemann, 1990).
39

None of our analysis has relied on agent experience or familiarity with the good. While

these may influence the agent’s true WTP for the good, they do not influence the incentive

properties of question format in the context in which it is being used. Nor have we relied on any

notion that agents learn about preferences and update them. Informational and incentive

properties of formats do play a role in updating of optimal response strategies. Indeed, it is

possible to recast some Bayesian models, such as the recent work of McLeod and Bergland

(1999), as Bayesian updating not with respect to preferences, but rather, with respect to

determining the optimal strategic response.

A number of elicitation formats commonly used in marketing research are currently

attracting considerable attention in environmental valuation, both for the hope that more

information can be collected from each agent (than can be collected with the binary discrete

choice format) and for the hope that these newer formats will have fewer problems than does a

binary discrete choice format. From an incentive perspective, this latter hope is likely to be

misplaced with respect to the two most common valuation situations in environmental

economics, the provision of public goods provided by the government and changes in a single

quasi-public good provided by the government. The generalization of the binary discrete choice

format in the directions used by marketing researchers causes it to lose its desirable incentive

properties. Further, as the number of goods that must be described in a survey increases the time

available to describe each good shrinks. For the introduction of new private goods, the

multinomial choice format may be close to incentive compatible from the perspective of

estimating marginal trade-offs between attributes as long as the perceived number of goods that

are likely to be provided is sufficiently large. This is because deviations from truthful preference

revelation are most likely to impact the scale parameter that drops out of marginal comparisons.
40

This fortunate occurrence is less likely to be true for estimating the total value of a good since

that calculation requires a consistent estimate of the true scale parameter.

Our work suggests that there are different natural underlying economic structures to

different valuation problems. The typical problem in environmental valuation is the

determination of the total value of a single good or a non-marginal change in a single good. The

strategic incentives facing agents confronting this problem may be the major force that has

moved researchers away from the open-ended and ranking formats toward binary discrete choice

formats. The typical problem in marketing is the determination of a trade-off between different

attributes of goods when many competing goods will be offered. Researchers in this area have

moved from open-ended and discrete choice questions, to ranking questions, and then to the

multinomial choice format.

A shift to the paired comparison or multinomial choice format has sometimes been

recommended as a means of reducing or eliminating the sensitivity of the estimate of the value of

a particular good to the sequence in which it was valued. However, this sensitivity is not a

problem of elicitation format. Attempts to get sequence effects to go away by shifting to a

question format that explicitly involves multiple goods are misguided. One of the major

differences between private goods and public goods is that, for the former, agents themselves

largely determine the order in which they obtain information about goods and make purchases of

them. For public goods, the government through its control of the agenda determines the order in

which projects are considered. Sequence effects (Carson, Flores, and Hanemann, 1998) are

inherent to sequential decision making and because substitution effects enter WTP calculations

much differently than they do demand calculations, sequence effects are apt to be large. Flores
41

(1994) shows that the classic agenda control problem can be rewritten in terms of WTP and

WTA sequences.

In closing, a remark on the term hypothetical, frequently affixed as an adjective in front

of the word survey, is in order. In a famous and often cited remark on the early use of surveys for

environmental valuation, Scott (1965) bluntly states: “Ask a hypothetical question and you get a

hypothetical answer.” Hypothetical as used here seems to imply that the responses are to an

“imaginary” inconsequential situation, and, as such, the responses will have no influence on any

relevant decision. From an economic perspective nothing can be inferred about respondent

preferences from asking such a question.

The term hypothetical, however, also means conjecture, counterfactual, and contingent.

This is the context usually used by researchers who ask preference questions. It is consistent with

our definition of a consequential survey but an incomplete one because we require the agent to

care about the alternatives and the agent to perceive that the agency will take the survey

responses into account in its decision making. Our suggestion is to eschew the use of the word

hypothetical in discussing preference questions in favor of consequential and inconsequential to

emphasize the conditions requisite for the application of economic theory.
42

References

Adamowicz, W. J. Louviere, and M. Williams (1994), “Combining Revealed and Stated

Preference Measures for Valuing Environmental Amenities,” Journal of Environmental

Economics and Management, 26, 271-292.

Adamowicz, W., J. Swait, P. Boxall, and J. Louviere (1997), “Perceptions Versus Objective

Measures of Environmental Quality in Combined Revealed and Stated Preference Models of

Environmental Valuation,” Journal of Environmental Economics and Management, 32, 65-

84.

Alberini, A. (1995), “Efficiency vs Bias of Willingness-to-Pay Estimates: Bivariate and IntervalData Models,” Journal of Environmental Economics and Management, 29, 169-180.

Alberini, A., B. Kanninen, R.T. Carson (1997), “Modeling Response Incentive Effects in

Dichotomous Choice Contingent Valuation Data,” Land Economics 73, 309-324.

Arrow, K., R. Solow, P.R. Portney, E.E. Leamer, R. Radner, and H. Schuman (1993), “Report of

the NOAA Panel on Contingent Valuation.” Federal Register 58, 4601-4614.

Bateman, I.J., I.H. Langford, R.K. Turner, and K.G. Willis (1995), Elicitation and Truncation

Effects in Contingent Valuation Studies,” Ecological Economics, 12, 161-179.

Bateman, I.J., A. Munro, B. Rhodes, C. Starmer, and R. Sugden (1997) “Does Part-Whole Bias

Exist?: An Experimental Investigation,” Economic Journal, 107, 322-332.

Bishop, R.C. and T.A. Heberlein (1979), “Measuring Values of Extra-Market Goods: Are

Indirect Measures Biased," American Journal of Agricultural Economics, 61, 926-930.

Bishop, R.C. and T.A. Heberlein (1990), “The Contingent Valuation Method,” in Economic

Valuation of Natural Resources: Issues, Theory, and Applications, R.L. Johnson and G.V.

Johnson, eds. (Boulder, CO: Westview Press).

Bergstrom, T., L. Blume, and H. Varian (1986), “On the Private Provision of Public Goods,”

Journal of Public Economics, 29, 2-49.

Boxall, P.C., W.L. Adamowicz, J. Swait, M. Williams, and J. Louviere (1996), “A Comparison

of Stated Preference Methods for Environmental Valuation,” Ecological Economics, 18, 243-

253.

Boyle, K.J., R.C. Bishop and M.P. Welsh (1985), “Starting Point Bias in Contingent Valuation

Bidding Games, “ Land Economics, 61, 188-194.

Boyle, K.J, F.R. Johnson, D. McCollum, and W.H. Desvousges, R.W. Dunford, and S.P. Hudson

(1996), “Valuing Public Goods: Discrete Versus Continuous Contingent-Valuation

Responses,” Land Economics, 72, 381-396.

Brookshire, D.S., B.C. Ives, and W.D. Schulze (1976), “The Valuation of Aesthetic

Preferences,” Journal of Environmental Economics and Management, 3, 325-346.

Cameron, T.A., (1988), “A New Paradigm for Valuing Non-Market Goods Using Referendum

Data: Maximum Likelihood Estimation by Censored Logistic Regression,” Journal of

Environmental Economics and Management, 15, 355-379.
43

Cameron, T.A. and D.D. Huppert (1991), “OLS versus ML Estimation of Non-Market Resources

Values with Payment Card Interval Data,” Journal of Environmental Economics and

Management, 17, 230-246.

Cameron, T.A. and J. Quiggin (1994), “Estimation Using Contingent Valuation Data from a

"Dichotomous Choice with Follow-up" Questionnaire,” Journal of Environmental

Economics and Management, 27, 218-234.

Carson, R.T. (1985), "Three Essays on Contingent Valuation," unpublished dissertation, University

of California, Berkeley.

Carson, R.T., N.E. Flores, K.M. Martin and J.L. Wright (1996), "Contingent Valuation and

Revealed Preference Methodologies: Comparing the Estimates for Quasi-Public Goods,”

Land Economics, 72, 80-99.

Carson, R.T., T. Groves, and M.J. Machina (1997), “Stated Preference Questions: Context and

Optimal Response,” paper presented at the National Science Foundation Preference

Elicitation Symposium, University of California, Berkeley.

Carson, R.T., N.E. Flores and W.M. Hanemann (1998), “Sequencing and Valuing Public

Goods,” Journal of Environmental Economics and Management, 36, 314-323.

Carson, R.T., W.M. Hanemann and R.C. Mitchell (1987). “The Use of Simulated Political

Markets to Value Public Goods.” Discussion paper 87-7, Department of Economics,

University of California, San Diego, March.

Carson, R.T., W. M. Hanemann, R.J. Kopp, J.A. Krosnick, R.C. Mitchell, S. Presser, P.A. Ruud,

and V.K. Smith (1994), "Prospective Interim Lost Use Value Due to DDT and PCB

Contamination in the Southern California Bight," report to National Oceanic and Atmospheric

Administration, September.

Carson, R.T. and D. Steinberg (1990), "Experimental Design for Discrete Choice Voter Preference

Surveys," in 1989 Proceeding of the Survey Methodology Section of the American Statistical

Association, (Washington: American Statistical Association, 1990).

Champ, P.A., R.C. Bishop, T.C. Brown, and D.W. McCollum (1997), “Using Donation

Mechanism To Value Nonuse Benefits From Public Goods,” Journal of Environmental

Economics and Management, 33, 151-162.

Chapman, R.G. nd R. Staelin (1982), “Exploiting Rank Ordered Choice Set Data Within the

Stochastic Utility Model,” Journal of Marketing Research, 19, 281-299.

Cronin, F.J. (1982), “Valuing Nonmarket Goods Through Contingent Markets,” report to U.S.

Environmental Protection Agency by Battelle Memorial Institute, Richmond Washington.

Cummings, R.G., G.W. Harrison and E.E. Rutström (1995), “Homegrown Values and

Hypothetical Surveys: Is the Dichotomous Choice Approach Incentive Compatible?",

American Economic Review 85, 260-266.

Cummings, R.G., S. Elliott, G.W. Harrison and J. Murphy (1997), “Are Hypothetical Referenda

Incentive Compatible?,” Journal of Political Economy 105, 609-621.
44

Cummings, R.G. and L. Osborne. (1998), “Does Realism Matter in Contingent Valuation

Surveys?” Land Economics, 74, 203-215.

Farmer, M.C. and A. Randall (1996), “Referendum Voting Strategies and Implications for

Follow-Up Open-Ended Responses,” paper presented at the annual U.S.D.A. W- 133

meeting, Jekyll Island, GA, February.

Farquharson, R. (1969), Theory of Voting. (New Haven: Yale University Press).

Flores, N.E. (1994), "The Importance of Agenda and Willingness to Pay." Paper presented at the

1994 Public Choice Society Meetings, Austin, TX, April.

Gibbard, A. (1973), “Manipulation of Voting Schemes: A General Result,” Econometrica 41,

587-601.

Green, J.R. and J.J. Laffont (1978), “A Sampling Approach to the Free Rider Problem,” in Agnar

Sandmo, ed., Essays in Public Economics (Lexington, MA: Lexington Books).

Greenhalgh, C. (1986), “Research for New Product Development,” in Consumer Market

Research Handbook, 3rd ed., R.M. Worcester and J. Downham, eds. (Amsterdam: NorthHolland).

Grether, D.M and C.R. Plott (1979), “Economic Theory of Choice and the Preference Reversal

Phenomenon,” American Economic Review, 69, 623-638.

Groves, T. (1973), “Incentives in Teams,” Econometrica, 41, 617- 631.

Groves, T., R. Radner, and S. Reiter, eds. (1987), Information, Incentives, and Economic

Mechanisms: Essays in Honor of Leonid Hurwicz (Minneapolis: University of Minnesota

Press).

Haab, T.C. and K.E. McConnell (1998), “Referendum Models and Economic Values:

Theoretical, Intuitive, and Practical Bounds on Willingness to Pay,” Land Economics, 74,

216-229.

Haab, T.C. and K.E. McConnell (1997), “Referendum Models and Negative Willingness to Pay:

Alternative Solutions,” Journal of Environmental Economics and Management, 32, 251-270.

Haab, T.C., and J.C. Huang, and J.C. Whitehead (1999). “Are Hypothetical Referenda Incentive

Compatible?: A Comment,” Journal of Political Economy, 107, 186-196.

Hanemann, W.M. (1984a), “Discrete/Continuous Models of Consumer Demand,” Econometrica,

52, 541-561.

Hanemann, W.M. (1984b), “Welfare Evaluations in Contingent Valuation: Experiments with

Discrete Responses,” American Journal of Agricultural Economics, 66, 335-379.

Hanemann, W.M., and B. Kanninen (1999), "The Statistical Analysis of Discrete-Response." In

Valuing the Environment Preferences: Theory and Practice of the Contingent Valuation

Method in the US, EC and Developing Countries, edited by Ian Bateman and Ken Willis

(Oxford: Oxford University).
45

Hanemann, W.M., J. Loomis, and B. Kanninen (1991), “Statistical Efficiency of Double

Bounded Dichotomous Choice Contingent Valuation,” American Journal of Agricultural

Economics, 73, 1255-1263.

Hausman, J.A., ed. (1993), Contingent Valuation: A Critical Assessment (Amsterdam: NorthHolland).

Hausman, J.A. and P. Ruud (1987), “Specifying and Testing Econometric Models for RankOrdered Data,” Journal of Econometrics, 34, 83-104.

Hensher, D.A. (1994), “Stated Preference Analysis of Travel Choice—The State of Practice,”

Transportation, 21, 107-133.

Hoehn, J. and A. Randall (1987), “A Satisfactory Benefit Cost Indicator from Contingent

Valuation,” Journal of Environmental Economics and Management 14, 226-247.

Holt, Charles A. (1986), “Preference Reversals and the Independence Axiom,” American

Economic Review, 76, 508-515.

Hurwicz, L. (1986), “Incentive Aspects of Decentralization,” in Handbook of Mathematical

Economics, vol. III, K.J. Arrow and M.D. Intrilligator, eds. (Amsterdam: North-Holland).

Inforsino, W.J. (1986), “Forecasting New Product Sales From Likelihood of Purchase Ratings

(with discussion),” Marketing Science, 5, 372-390.

Johannesson, M., B. Liljas, P.O. Johansson (1998), “An Experimental Comparison of

Dichotomous Choice Contingent Valuation Questions and Real Purchase Decisions, Applied

Economics, 30, 643-647.

Johnson, F.R. and W.H. Desvousges (1997), “Estimating Stated Preferences with Rated-Pair

Data: Environmental, Health, and Employment Effects of Energy Programs,” Journal of

Environmental Economics and Management, 34, 79-99.

Kahneman, D., P. Slovic, and A. Tversky (1982), Judgment under Uncertainty: Heuristics and

Biases (New York : Cambridge University Press).

Karni, E. and Z. Safra (1987), “Preference Reversal and the Observability of Preferences by

Experimental Methods,” Econometrica, 55 675-685.

King, G. (1989), Unifying Political Methodology: The Likelihood Theory of Statistical Inference,

(New York: Cambridge University Press).

Kriström, B. (1990), “A Non-Parametric Approach to the Estimation of Welfare Measures in

Discrete Response Valuation Studies,” Land Economics, 66, 135-139.

Kriström, B. (1997), “The Practical Problems of Contingent Valuation, in R.J. Kopp, W.W.

Pommerhene, and N. Schwartz, eds., Determining the Value of Non-Marketed Goods

(Boston: Kluwer).

Lankford, R.H. (1985), “Preferences of Citizens for Public Expenditures on Elementary and

Secondary Education,” Journal of Econometrics, 27, 1-20.

Louviere, J.J. (1994), “Conjoint Analysis,” in Handbook of Marketing Research, R. Bagozzi, ed.

(Oxford: Oxford University Press).
46

Lohmann, S. (1994), “Information Aggregation Through Costly Political Action,” American

Economic Review 84, 518- 530.

Lunander, A. (1998), “Inducing Incentives to Understate and to Overstate Willingness to Pay

within the Open-Ended and the Dichotomous-Choice Elicitation Format,” Journal of

Environmental Economics and Management, 35, 88-102.

Lupia, A. (1994), “Shortcuts Versus Encyclopedias—Information and Voting Behavior in

California Insurance Reform Elections,” American Political Science Review, 88, 63-76.

Machina, M.J. (1995), “Non-Expected Utility and the Robustness of the Classical Insurance

Paradigm,” Geneva Papers on Risk and Insurance Theory, 20, 9-50.

McConnell, K. E. (1990), “Models for Referendum Data: The Structure of Discrete Choice

Models for Contingent Valuation,” Journal of Environmental Economics and Management,

18, 19-34.

McDowell, I. and C. Newell (1996), Measuring Health: A Guide to Rating Scales and

Questionnaires, 2nd ed. (New York: Oxford University Press).

McFadden, D. (1974), “Conditional Logit Analysis of Qualitative Choice Behavior,” in

Frontiers in Econometrics, P. Zarembka, ed., (New York: Academic Press).

McFadden, D. (1994), “Contingent Valuation and Social Choice,” American Journal of

Agricultural Economics 76, 689-708.

McFadden, D. (1999) “Rationality for Economists?,” Journal of Risk and Uncertainty, 19, 73-105.

McLeod, D.M; Bergland, O. (1999), “Willingness-to-pay Estimates Using the Double-Bounded

Dichotomous-Choice Contingent Valuation Format: A Test for Validity and Precision in a

Bayesian Framework, Land Economics, 75, 115-125.

Mitchell, R.C. and R.T. Carson (1986), "The Use of Contingent Valuation Data for Benefit-Cost

Analysis in Water Pollution Control," report to the U.S. Environmental Protection Agency,

September 1986.

Mitchell, R.C. and R.T. Carson (1989), Using Surveys to Value Public Goods: The Contingent

Valuation Method. Baltimore: Johns Hopkins University Press.

Moulin, H. (1994), “Social Choice,” in Handbook of Game Theory with Economic Applications,

R.J. Aumann and S. Hart, eds. (Amsterdam: North- Holland).

Neil, H., R.G. Cummings, P.T. Ganderton, and G.W. Harrison (1994), “Hypothetical Surveys

and Real Economic Commitments,” Land Economics 70, 145-154.

Payne, S. (1951), The Art of Asking Questions (Princeton: Princeton University Press).

Polasky, S., O. Gainutdinova, and J. Kerkvliet (1996), “Comparing CV Responses with Voting

Behavior: Open Space Survey and Referendum in Corvallis Oregon,” paper presented at the

annual U.S.D.A. W- 133 meeting, Jekyll Island, GA, February.

Popkins, S. (1991), The Reasoning Voter (Chicago: University of Chicago Press).
47

Posavac, S.S. (1998), “Strategic overbidding in contingent valuation: Stated economic value of

public goods varies according to consumers expectations of funding source,” Journal of

Economic Psychology,19, 205-214.

Randall, A. (1994), “A Difficulty with the Travel Cost Method,” Land Economics, 70, 88-96.

Randall, A. (1996), “Calibration of CV Responses: Discussion,” in The Contingent Valuation of

Environmental Resources: Methodological Issues and Research Needs, D.J. Bjornstad and J.

R. Kahn (Brookfield, VT: Edward Elgar).

Randall, A., B.C. Ives, and C. Eastman (1974), “Bidding Games for Valuation of Aesthetic

Environmental Improvements,” Journal of Environmental Economics and Management 1,

132-149.

Richer, J. (1995), "Willingness to Pay for Desert Protection," Contemporary Economic Policy,

13, 93-104.

Romer, T. and H. Rosenthal (1978), “Political Resource Allocation, Controlled Agendas, and the

Status Quo,” Public Choice, 33, 27-43.

Rowe, R.D., W.D. Schulze, and W. Breffle (1996). “A Test for Payment Card Biases,” Journal

of Environmental Economics and Management 31, 178-185.

Samuelson, P.A. (1954), “The Pure Theory of Public Expenditure,” Review of Economics and

Statistics 36, 387- 389.

Satterthwaite, M. (1975), “Strategy-Proofness and Arrow Conditions: Existence and

Correspondence Theorems for Voting Procedures and Welfare Functions,” Journal of

Economic Theory 10, 187-217.

Swait, J. and J.J. Louviere (1993), “The Role of the Scale Parameter in the Estimation and Use of

Generalized Extreme Value Models,” Journal of Marketing Research, 30, 305-314.

Scott, A. (1965), “The Valuation of Game Resources: Some Theoretical Aspects,” Canadian

Fisheries Report 4, 27-47.

Seip, K. and J. Strand (1992), “Willingness to Pay for Environmental Goods in Norway: A CV

Study with Real Payment,” Environmental and Resource Economics, 2, 91-106.

Smith, V.K. and W.H. Desvousges (1986), Measuring Water Quality Benefits (Boston: Kluwer).

Sudman, S., N.M. Bradburn, and N. Schwarz (1996), Thinking About Answers: The Application

of Cognitive Processes to Survey Methodology (San Francisco: Jossey-Bass Publishers).

Sugden, R. (1999) “Alternatives to Neo-classical Theory of Choice,” in Valuing Environmental

Preferences: Theory and Practice of the Contingent Valuation Method in the USA, EC, and

Developing Countries, edited by I.J. Bateman and K.G. Willis. (Oxford: Oxford University

Press).

Tversky, A., P. Slovic, and D. Kahneman (1990), “The Causes of Preference Reversal,”

American Economic Review, 80, 204-217.

Varian, H. (1992), Microeconomic Analysis, 3rd. ed. (New York: Norton).