Incentive and Informational Properties of Preference Questions 1

Richard T. Carson Theodore Groves Mark J. Machina Department of Economics, 0508 University of California, San Diego La Jolla, CA 92093 rcarson@ucsd.edu

Draft February 2000

1 E arlier versions of this paper were presented in Oslo as a plenary address to the European Association of Environmental and Resource Economics, as an invited paper at the Japanese Form on Environmental Valuation meeting in Kobe, and at a NOAA conference on stated preference methods. Support of U.S. Environmental Protection Agency cooperative agreement R-824698 in carrying out the research reported on in this paper is gratefully acknowledged. The views expressed are those of the authors and not necessarily those of the U.S. Environmental Protection Agency.1

Introduction Businesses and governments frequently use surveys to help determine the relevant public’s preferences toward different actions that could be taken. Applications are particularly common in environmental valuation (Mitchell and Carson, 1989), health care (McDowell and Newell, 1996), marketing (Louviere, 1994), political science (King, 1989) and transportation (Hensher, 1994). As long as the economic agents (hereafter, “agents”) being surveyed believe that the survey responses might influence actions taken by businesses and governments (hereafter, “agency”), the standard economic framework suggests that the agents should respond to the survey in such a way as to maximize expected utility. Given the billions of dollars spent annually on surveying and the frequently voiced concern that marketing surveys determine the fate of products and that major political decisions are largely poll-driven, the position of many economists that survey responses should be ignored as a source of information on preferences is somewhat surprising. These economists seem to regard survey responses as either completely meaningless because they are answers to hypothetical questions or as completely useless because agents will respond strategically. The first reason violates the standard rationality condition assumed of agents if agents believe that agency decisions are being made at least in part on the basis of the survey responses. The second reason stops short of the more relevant question: what are the strategic incentives and how should they influence responses? In this paper, we systematically explore implications of the economic maximization framework for the behavior that one should expect to see from rational agents answering survey questions concerning preferences. The economic literature on neoclassical choice theory and mechanism design (Hurwitz, 1986; Groves, Radner and Reiter, 1987; Varian, 1992) provides the2

theoretical foundation for our work. This body of work can be contrasted with those who reject this framework in favor of other psychologically based theories (e.g., Kahneman, Slovic and Tversky, 1982; Sugden, 1999, McFadden, 1999). We believe that at least some of the evidence put forward in favor of those theories, particularly with respect to what differences should be expected with respect to asking questions using different response modes, has been incorrectly interpreted. We have endeavored here to put forth our results in an intuitive non-mathematical fashion as we hope fundamentally that our models represent a common sense approach to thinking about how agents should view preference questions. In the model informally presented here, agents are assumed to decide (1) whether they care about how the outcome might be influenced by the answers they provided, (2) whether the aspects of the scenario described are plausible, and (3) how the survey results are likely to be used. Judgements respecting these assumptions need not be elaborately or explicitly articulated any more than most judgements in life are. These three assumptions combined with the basic maximizing rationality assumption are capable of yielding a surprisingly rich picture of the manner in which agents should respond to survey questions. A major reason that estimates of economic value from surveys are looked upon with suspicion by economists is a number of results that seem inconsistent with respect to economic intuition. These anomalous results have been interpreted by different researchers as evidence of (a) the hypothetical nature of the question, (b) strategic behavior 2 , or (c) preferences that are

2 The possibility of strategic misrepresentation of preferences has long been seen as one of the central problems in public economics. Samuelson (1954) argued: “It is in the selfish interest of each person to give false signals to pretend to have less interest in a given collective activity than he really has.” He made specific reference to the possibility of strategic behavior with respect to the use of surveys. Samuelson’s admonition, repeated in many textbook discussions of public goods, had a profound effect on how many economists view the survey questions. The mistaken inference made by many from this admonition was to equate strategic behavior with lying. As the term is used in the modern mechanism design literature in economics, strategic behavior is merely synonymous with a rational agent maximizing (broadly defined) self-interest. Mechanism design theory has shown that the optimal strategic behavior for agents in many instances is to truthfully reveal their preferences. Whether this is the case or3

either ill defined or inconsistent with economic theory. In attempting to systematically categorize these anomalies it becomes immediately apparent that there is an antecedent question: does a survey question need to meet certain conditions before it should be expected to produce results that are consistent with economic theory? This question turns out to be relatively easy to address from the standpoint of economic theory. First, the agent answering a preference survey question must perceive responses to the survey question as potentially influencing agency action. Second, the agent needs to care about

what the outcome of that action is. 3 We will term surveys that meet these two basic criteria as consequential

survey questions and those that don’t as inconsequential survey questions. In more formal terms, we can state the following:

Consequential and Inconsequential Preference Survey Questions : A. If the survey results are seen by the agent as potentially influencing agency actions and the agent cares about the outcome of that action, then the agent should treat the survey question as an opportunity to influence those actions. In the case of a consequential survey question, standard economic theory applies and the response to the question should be interpretable using mechanism design theory concerning incentive structures. B. If the survey responses are not seen as having any influence on agency decisions or the agent is indifferent to all possible outcomes of the agency decision, then all possible responses by the agent will be perceived as having the same influence on the final outcome. In this case of an inconsequential

survey question, economic theory makes no predictions about the nature of the responses to the survey given by the agent.

not depends upon the particular format of the preference question asked and other aspects of the scenario, including the type of good involved. 3 For instance, a non-smoker may not care about the addition of a new type of cigarette with a much lower nicotine level and a higher price to the current cigarette choice set. Confusion often exists over the magnitude of the possible change in utility from agency action and the incentives the agent faces in the response given to the question. The size of the utility change generally does not influence the incentive structure of the question as long as there are differences in utility levels between different agency actions. The size of the utility change can influence agent participation in the survey.4

Most preference survey questions asked by businesses and governments meet the two basic criteria for being a consequential survey question, and hence, should be interpretable in

economic terms. 4 There are, however, many preference survey questions that do not meet these

criteria. While most of these inconsequential survey questions could be characterized as issuing from psychology lab exercises with undergraduates, there are plenty of real world examples. 5 It is pointless to try to explain apparent economic anomalies in inconsequential survey questions because any response to such a question has the same effect on the agent’s utility. We are formally rejecting the notion sometimes advanced by proponents of the use of preference survey questions, which if a respondent perceives no gain or loss from how a preference survey is answered then that respondent will truthfully answer the question. While such an assumption may indeed be true, there is no basis in economic theory to either support or deny it. Among questions meeting these two criteria for being consequential to the agent, we examine five key issues which should illustrate both the power and limitations of economic theory to explain a large body of empirical evidence related to the performance of survey questions under particular conditions. First, we look at the properties of binary discrete choice questions under different circumstances. In particular, we examine whether binary discrete choice questions are incentive compatible in the sense that truthful preference revelation represents an optimal (and the dominant) strategy for the agent. The empirical evidence suggests that such questions often work well: they predict actual behavior quite closely and they are

4 Marketing research firms, in particular, face a constant battle between asking questions to only those who are currently using a product category and trying to reach the larger and harder to identify population of all potential users. For public goods provided via taxation, the situation is generally easier. Even if the respondent does not care whether the good is provided at zero cost, the respondent does care about its provision if the tax cost is positive. 5 Inconsequential preference questions can most often be identified by having one or more of the following identifying characteristics: (a) being asked of a population or at a location that is unlikely from the perspective of an agency seeking input on a decision, (b) providing few if any details about the goods and how they would be provided, (c) asking about goods that are implausible to provide, and (d) providing prices for the goods that are implausible.5

sensitive to factors such as the scope of the good being valued. However, there are instances where such questions perform quite badly. Second, we consider the reasons responses to repeated binary discrete choice questions ( e.g., double-bounded dichotomous choice) by the same respondent are often inconsistent with each other. We also consider what information might be provided to the agent by the second choice question in this section. Third, we look at whether binary discrete choice questions and open-ended continuous response questions should produce similar estimates of statistics such as mean or median willingness to pay (WTP). In this section, we pay particular attention to the issue of what role, if any, information on cost should have on reported WTP values. Fourth, we consider the implication of moving from valuation of a single good to valuation of multiple goods, first in the context of the sequence of pair comparisons and then in the context of the increasingly popular multinomial choice questions. To begin to understand these issues, it is necessary to first confront what we have termed the face

value dilemma.

The Face Value Dilemma Economists tend to either reject preference survey results out of hand or treat the answers as truthful responses to the question asked. We term this latter behavior as taking the survey

answers at face value . The two positions are not unrelated as both are result-oriented rather than process-oriented; many economists who reject the use of survey questions do so because the results are anomalous if taken at face value. We believe that either rejecting the usefulness of the preference survey answers or taking them at face value is likely to be wrong in many

circumstances even when the two basic criteria for a consequential preference survey question

have been met.6

The face value assumption can be formally defined as “the assumption that respondents always truthfully answered the specific preference question intended to be asked”. There are two key parts of this assumption: (a) respondents always truthfully reveal preferences, and (b) the specific question being asked is the one being answered. Note that (a) and (b) are both very strong assumptions. While the mainstream economic position is that (a) is dubious due to the strategic behavior, this assumption is routinely maintained in marketing research, political polling, psychology, sociology and other fields heavily dependent on survey research. In contrast, while economists who do use survey results routinely seem to believe (b), survey researchers have shown this to be a dubious assumption (Sudman, Bradburn, Schwarz. 1996). Interpreting responses to survey questions appropriately requires consideration of the possibility that neither part of the face value assumption maybe be true. For truthfully revealing preferences, objections that agents may be responding strategically are insufficient to reject the

use of consequential preference survey questions, as it may be in the respondent’s strategic interest to truthfully reveal their preferences under some question formats in particular contexts. 6 With respect to the decoupling of question and answer, the survey research community’s usual rationale for the possibility that respondents may answer a different question than the one asked, is simply that respondents may not understand the question actually asked and instead answer the question that they think is being asked. Part of the survey designer’s art lies in the crafting of language that elicits the answer to the question that the researcher intended to ask (Payne, 1951). From the perspective of preference survey questions for non-marketed goods or new consumer products, this issue needs to be taken particularly seriously since the development

6 Furthermore, under other question formats, the expected direction of the bias in responses can be theoretically predicted in some instances and empirically confirmed. In such cases, the results, even if biased, may be useful and often sufficient for agency decision making (Hoehn and Randall, 1987).7

of questionnaires describing such goods is among the more difficult of survey design tasks; andmost economists developing such surveys have little formal training in survey design. The preeminent issue here is that if survey responses are to be taken at face value, the question as written should elicit the answer to the question intended to be answered by the designer with all the conditions with which the designer wanted it answered. If this does not happen the results can easily be taken as implying violations of economic theory, when what has in reality happened is that agents have answered a different question. 7 A further issue should be raised which concerns asking preference questions with implausible premises, for example, asking a binary discrete choice question with an implausibly high or low cost for providing the good. In such instances, respondents are likely to substitute what they consider to be the expected cost for the good and answer on that basis. Another easily recognized variant of this issue concerns implausible characteristics of the good provided, such as an assertion that a risk reduction program would be 100% successful, an assertion which is likely to be discounted by agents. More complicated variants of the issue manifest themselves when a respondent is given information at various points in a survey that is inconsistent. Examples include providing two different cost numbers in the double-bounded dichotomous-choice elicitation format and asking respondents about the provision of different levels of the same public good at different places in the survey. A key implication of this line of argument is that

7 For example, if a subset of agents does not believe that the good can be provided in any amount, these agents should be insensitive to the quantity (scope) of the good to be provided even though they may place a positive value on it. Divergences between the intended and answered question will always occur to a greater or lesser degree. The survey designer should endeavor to minimize them and the analyst should determine how they need be taken into account in order to arrive at reliable estimates. It should be noted that there is nothing unique about the use of stated preference data with respect to this issue. Most economic analyses of revealed preference data use objective indicators of good attributes to predict agent choices even though using agent perceptions of them can usually be shown to provide better predictions (Adamowicz et al. , 1997). Estimates of the value of a statistical life based upon hedonic wage equations have always been plagued by the need to make the demonstrably false assumption that agents were aware of the objective risk level used as a predictor variable in the equation.8

there are likely to be limits to the range of preference questions that a researcher can expect to have respondents answer. Survey questions can extend the range of goods and their attributes, including price, considerably beyond what agents have previously experienced; but any counterfactual scenarios must be credible portraits of possible future outcomes.

A Simple Typology of Elicitation Formats The truthful preference revelation part of the face value assumption implies different conditions for different elicitation formats. This can most easily be seen by noting, that from a strategic perspective, all of the standard question formats can be shown to be generalizations of the single binary discrete choice format (Figure 1). Under this format, the respondent is told about two different alternatives and is assumed to pick which of the two alternatives provides the highest level of utility. As we discuss at length below, this assumption may be justified under

some sets of conditions but not others.

FIGURE 1 Single Binary Choice “One-Shot Choice”

Sequence of Binary Choices *Assumes Independence Across Choices Multinomial Choice One-Shot Choice, k > 2 Alternatives Equivalency (Valuation) * Assumes a Continuous Variable (e.g . money)

Sequence of Multinomial Choices * Assumes ‘Independence’ Across Choice SetsSequence of Intensity-ofPreference Questions *Assumes Cardinal Utility9

There are three basic ways a single binary discrete choice question can be generalized. These are the open-ended matching type question, a sequence of binary choice choices, and the multinomial choice question. Matching questions, rather than ask for a choice between two alternatives, drops an attribute level (typically cost) of the second choice and asks the agent to provide the quantity of the attribute level that would make the agent see the two choices as equivalent in terms of utility to the agent. A sequence of binary choice questions adds the assumption that the agent answers each pair of choice independently to the assumption made of the single binary discrete choice. A number of different formats can be shown to be strategically manifestations of the sequence of binary discrete choice questions including the popular double bounded dichotomous choice format in contingent valuation (Hanemann, Loomis, and Kanninen, 1991) and the complete ranking of alternatives popular in marketing which can always be exploded into a set of binary paired comparisons if the independence assumption holds (Chapman and Staelin, 1982). Another commonly used variant of the sequence of binary choices asks agents to “rate” one choice relative to the other on a numeric scale ( e.g. , 1 to 10) and exploits the information revealed about preference intensity (Johnson and Desvousges, 1997). This adds the assumption of cardinal utility. A multinomial choice question adds the assumption that the agent picks the most preferred out of k > 2 alternatives. A popular variant of this format, a sequence of multinomial choice sets (Louviere, 1994) adds the same assumption that a sequence of binary choice questions does, independence in responses across the choice sets. For each of these questions formats it is possible to look at the divergence between the face value response and the strategic response. It is also possible to look at differences in the set of information conveyed by a particular elicitation format. Because the different elicitation formats are generalizations of the binary discrete choice format and because it can be shown that the10

binary discrete choice format has different strategic properties in different context we start with

an examination of that format.

Binary Discrete Choice Preference Questions A single binary discrete choice question between two alternatives, one typically being the status quo, is one of the most commonly used preference elicitation formats. It has a long history of use in survey research, and most other discrete choice and ranking formats can be easily shown to be generalizations of it. Bishop and Heberlein (1979) showed that this format could be used along with the random assignment of respondents to different monetary costs to recover the distribution of willingness to pay or willingness to accept compensation (WTA). Later papers by Hanemann (1984a, 1984b) formally worked out the utility theoretic approach from a random utility perspective (McFadden, 1974); and Cameron (1988) provided a purely statistical approach of tracing out the latent (unobserved) WTP or WTA variable in a manner similar to dose response experiments in biology or medicine. McConnell (1990), Kriström (1997), Haab and McConnell (1997; 1998) and Hanemann and Kanninen (1999) provide comprehensive examinations of the statistical issues involved in using the binary discrete choice format. While we will generally ignore the substantive issues raised in these papers with respect to the estimation process, we do note some of the implausible estimates that in the literature appear to be the result of failing to adequately model the data and incorporate sensible restrictions implied by economic theory. Much of the attention focused on the binary discrete choice elicitation format in recent years is due to the NOAA Panel on Contingent Valuation’s (Arrow et al. , 1993) recommendation for its use as a consequence of its well-known property of being incentive compatible in some circumstances. Indeed, one of the core results in mechanism design theory independently derived11

by Gibbard (1973) and Satterthwaite (1975) is that no response format that allows for more than a binary response can be incentive compatible without assuming restrictions on the realm of allowable agent preferences. However, the Gibbard-Satterthwaite result is essentially a negative one—no response format with greater than a binary choice (including all multinomial and continuous response formats) can be incentive compatible without restrictions on preferences. This result does not say

that all or even any binary discrete choice formats are incentive compatible; only that this is the only response format that is potentially incentive compatible. It has long been known that in some settings that the binary discrete choice format is incentive compatible (Farquharson, 1969). The best-known examples are political races with only two candidates and binding (approve/disapprove) referendums with a plurality (usually majority or two-thirds approval) vote requirement. The binding referendum is a useful departure point for our discussion and the NOAA Panel references this mechanism before their recommendation to use a binary discrete choice format in contingent valuation (CV) surveys. The first question is whether it is the binding nature of the referendum that makes it incentive compatible. Carson, Groves, and Machina (1997) consider an advisory referendum

vote. 8 They show that replacing the binding plurality vote requirement with the weaker assumption that, over some range, the government is more likely to undertake an action the larger the percentage in favor. 9 The plurality vote requirement is a special case of this

8 Many well-known referendums are technically advisory referendum. For example, Norway’s vote on whether to join the European Union (EU) was an advisory referendum. Some observers believed that if the vote in favor were only a very slim majority, that the government would not join the EU, however, if a substantial majority favored joining then the government would join the EU. 9 It is necessary to assume that agents believe they have only influence locally around the amount they are asked if this response function is considered to cover the case where the amounts agreed to are summed. We are indebted to Pere Riera for this observation.12

assumption with the knifed-edged decision rule that any vote less than the required plurality for the new (“yes”) alternative results in the default (“no”) alternative being implemented. The second question is: does substituting an advisory survey for an advisory referendum alters the incentive properties of the mechanism? Green and Laffont (1978) have shown that any economic mechanism of the types being considered in this paper can be implemented using a sampling approach rather than complete participation. Thus, we come to the following:

Result: It is possible to replace the binding nature of an incentive compatible referendum with the more general assumption that the agency is more likely to undertake the action the higher percent in favor. It is also possible to substitute a survey of the public for a vote of the public on the issue. Neither of these changes, alone or together alter the original incentive structure of the binding referendum. A small number of CV studies ( e.g., Carson, Hanemann, and Mitchell, 1987; Polasky, Gainutdinova and Kerkvliet, 1996), which have compared survey estimates to the vote on actual binding referendums, have found the two to be quite close. A very large body of evidence from polling on referendum suggests that surveys taken close to an election generally provide quite good predictions of actual referendum votes. 10 It is important to note, however, that it is not casting the preference question as a referendum that provides its desirable incentive properties. It is the cast of the preference question in terms of being able to influence a government decision with a binary favor/not favor format. Two key assumptions have been made in the discussion of the preceding sequence of mechanisms. The first assumption is that the agency ( i.e ., government) can compel payment for a

10 Predicting an actual election vote from a survey involves two key difficulties unrelated to whether agents truthfully reveal their preferences in surveys. The first is that the information set the voter uses on election day may have changed from the one at the time of the survey due to activities such as political advertising and media coverage. It is this factor that makes surveys taken close to an election generally more accurate than surveys taken at some distance from the election. (The dynamics of the information process are such that the proponents of the measure are usually able to initially put out a largely unopposed positive message. As opponents slowly start their negative campaign, support for the measure falls over time.) The second is predicting who is going to actually vote. The characteristics of a good random sample of the public may be substantially different from the characteristics of the sample of the public that actually votes.13

good if provided. The second assumption is that only a single issue is involved. Relaxing the first assumption destroys the incentive properties of what we will call the referendum—advisory referendum—advisory survey (RARAS) mechanism. To see this, consider the case where a charitable organization wants to provide a public good via voluntary contributions. A “yes” response to a binary discrete choice survey question of the form: “would you contribute $X to a fund to purchase the specified good if we started the fund?” will encourage the charitable organization to undertake the fundraising effort. Upon mounting the fundraising effort, the optimal strategic response of an agent who wants the public good will be to contribute less than her maximum willingness to pay for the good and in many instances to contribute nothing. 11 The preferred strategy is to sit back and wait to see if the good is provided without her contribution. This is the classic free riding behavior which economists have long seen as perhaps the fundamental problem with the provision of public goods. What is interesting in this case is that the same incentive structure which should cause free riding with respect to the actual contributions should induce respondents in a survey to over pledge because doing so helps to obtain the later opportunity to free ride. A number of empirical studies confirm the large predicted divergence between survey-based predictions of contributions and actual contributions ( e.g ., Seip and Strand, 1992; Champ et al. , 1997). Switching to the case of introducing a new private good does not improve the incentive situation. As long as there is any positive probability of wanting the new good at the stated price, the respondent should say, “yes—would purchase.” The agent’s logic is that such a response will encourage the company to produce the good, with the agent being able to decide later whether to

11 In many charitable fundraising efforts, the quantity of the good provided is increasing in the amount of money raised. In such a case, it may be optimal for a (non-pivotal) agent who desires the good to contribute at a positive amount toward its provision (Blume, Bergstrom and Varian, 1986).14

purchase. Since increasing the agent’s choice set in a desirable way increases utility, the optimal response is “yes.” Folk wisdom from the marketing research literature supports the notion that consumers overstate their purchase proclivities for new products (Greenhalgh, 1986). Evidence from experiments in economics (Cummings, Harrison, and Rustöm, 1995; Johannesson, Liljas, and Johansson, 1998) also supports this conclusion. The marketing research approach has tended to either shift to a different measurement scale such as the probability of purchasing (Inforsino, 1986) or to ask about more than one good (Louviere, 1994). There is some irony in this result as it has so often been said that if standard CV elicitation formats did not work well for private goods then they would not work for pure public goods that are not bought and sold in the marketplace. The familiarity argument that is so often heard in support of doing experiments with private goods to learn about how CV is likely to work in the best case scenario (Neil et al., 1994) is misguided. Examined in this light, the introduction of a new private good is one of the worst-case scenarios for a binary discrete choice question. It should not be surprising that the binary discrete choice format, which while initially seeing usage in marketing research, is now rarely used. The ability of the agency to coercively collect payment for the good is the property that causes the agent to try to influence the agency’s decision in the desired direction taking account of both the cost and the benefits of the action to the agent. 12 Voluntary contributions allow for

12 It is interesting to ask whether it is the two-step nature of a survey followed by a contribution/purchase that leads to the survey question not being incentive compatible. The answer is no. Consider the situation whereby the only way a public good can be provided is if it obtains the requisite plurality vote in a referendum and the legislature gets to decide whether to put the issue on the ballot for a vote. The legislature does not want to waste the public’s time putting on propositions to vote on if they stand little chance of passing. The legislature (or the measure supporters) commissions a survey to determine the likely fraction of the public that would vote in favor of the measure. The only consistent responses (given no change in the information set) to the survey and actual referendum vote are “yes” to both the survey and the referendum or “no” to both the survey and referendum. For those in favor of the measure, the only way to get the good is to get the referendum put on the ballot and have the measure passed. “Yes” responses to both opportunities increase the chance of both. For those opposed to the measure, saying “yes” to the survey increases the chance that it will get put on the ballot, which in turn increases the chance that the agent will have to pay for the good, even though the good is not worth the cost to the agent if provided.15

the possibility that the survey response encourages the fund-raising effort to be undertaken, and hence, the possibility of free riding during the actual fund-raising effort. Thus, agents who want the good provided should say “yes” (would contribute) to the survey. In turn, it will be optimal for some of these agents to free ride in the expectation that other agents would contribute enough to provide the good. In this case, an initial survey “yes” response helps to set up the later opportunity to free ride with respect to the actual contribution. For the private goods case, a “yes” response (would purchase) to the survey encourages the production of the good while the agent gets to decide later whether to purchase the good. Thus, if the agent anticipates any positive probability of wanting to purchase the good, then a “yes” response is optimal. If the agent anticipates that the good will be offered irrespective of the responses given by agents but the agent perceives that the responses may influence the price of the good, then it is optimal for the agent to appear more price sensitive than is actually the case. This result is often seen in marketing research where agents have been found to more price elastic in surveys than in actual market purchases. The only problem with these cases from the perspective of economic theory is not whether there should be a divergence between actual behavior and the survey estimate, but rather, whether the magnitude of the divergences empirically observed should be even larger. There are other interesting implications of the lack of incentive compatibility for binary discrete choice survey questions for voluntary contributions and the introduction of new private goods with respect to other anomalies such as insensitivity to the scope of the goods being valued. For instance, as long as the good is potentially desirable it is optimal to say “yes” to the survey question. The scope of the good and its cost do not influence this decision unless the good becomes so small that even if at a zero cost it is not desired or if the cost becomes so high that it16

would never be purchased. In both of these later instances, either a “yes” or a “no” response by the agent will have the same effect on the agent’s utility.

If the binary choice is between two different forms of a quasi-public or private good, then desirable incentive properties can be restored as long as only potential users are interviewed. 13 To see this, consider the classic case of a campsite. At present the campsite is unimproved and has a low entrance fee (possibly $0). The alternative is to improve the campsite and increase the entrance fee. The agent should now choose the status quo campsite price/quality combination or the alternative campsite price/quantity combination to maximize utility. This binary choice can be shown to have identical properties to the RARAS survey mechanism. The property that the mechanism needs to be incentive compatible is the ability of the agency to force one of the alternatives on a particular agent irrespective of that agent’s preferences in a situation where the agent’s utility is influenced by the agency decision. Two important caveats should be kept in mind. First, in this situation the total number of times the good will be used under the alternative is endogenous. In our campsite example, if the higher quality-price campsite alternative provides more utility than the status quo, the anticipated number of visits to that campsite under that alternative may be larger or smaller than under the status quo. Second, for agents whose probability of use of the good does not differ between the two configurations, any response has the same impact on the agent’s utility. This problem is not usually seen because most recreational surveys are either done on site or from lists of users. Marketing researchers typically screen out non-users of a product class before asking preference questions. 14 The risk in both

13 Quasi-public goods are those provided by the government but for which it is possible to exclude members of the public from using. This exclusion can occur in terms of charging a price to use the resource, having the agent spend money or time to use the resource or by having the resource effectively bundled as an attribute of a privately purchased product. Common examples include government campgrounds and houses located on public lakes.

14 There are exceptions. Boxall, et al. (1996), for instance, ask hunters in Alberta about two different management/cost regimes for a specific area that few currently hunted in and few were likely to hunt in with the17

instances is that focusing on current users of the good will miss those who would likely use the good if its quality/price attributes were changed. This choice between two configurations of a good works for public goods and private goods too, irrespective of the nature of the payment obligation, as long as the agent desires the good at no cost. To see this, consider a private charity that wanted to build one of two different monuments in the center of town. The charity conducts a survey of the public to determine which monument is preferred and the higher the level of support for a particular monument the more likely that monument will be built. The agent should pick the preferred monument since this increases the agent’s utility more than the alternative monument and neither monument imposes any cost on the agent. Our favorite example of a private good question is the bar owner that surveys patrons and asks whether they would prefer to have the bar’s sole draft beer, currently a domestic brand priced at $1, switched to an imported brand at $2. The bar patron should pick the import only if having that alternative available provides more utility than the domestic. Note that the number of beers that will be purchased is not revealed by the agent’s choice and could go up

or down. Table 1 summarizes the incentive properties of binary discrete choice questions by the type of good and the payment characteristics. In this case we have assumed that the agent would desire the good if there was no cost, otherwise the incentive properties of the question are undefined. What is striking is that anomalies with respect to a divergence between estimates based on stated preferences and estimates based on behavior are heavily concentrated in the two cases that are not incentive compatible.

alternative management scheme. In this instance, the contingent valuation estimate was dramatically larger than the travel cost estimate, something that is fairly unusual in comparisons between the two approaches for quasi-public goods (Carson et al., 1996). When the estimate of the change in the probability of use is used to scale the CV estimate, the two approaches result in quite similar estimates.18

Table 1: Incentive Properties of Binary Discrete Choice Questions Type of Good Incentive Property New public good with coercive payment Incentive compatible New public good with voluntary payment Not incentive compatibleIntroduction of new private or quasipublic good Not incentive compatible Choice between which of two new public goods to provide Incentive compatibleChange in an existing private or quasipublic good Incentive compatible but choice does not reveal information about quantities

The second key assumption in the discussion of the RARAS mechanism is that is that of a single up-down vote on a single issue. It is also not possible to relax this condition and there

are several common instances where it is violated. The best-known ones are the rules for school bond referendums in many areas (Romer and Rosenthal, 1978; Lankford 1985). The school board gets to propose the level of educational inputs and the tax rate. However, if the referendum is voted down, the school board can only bring up another referendum measure with a level of educational inputs and a tax rate that is lower than those voted down but higher than the default status quo. A respondent who prefers the initially offered bundle to the status quo may nonetheless have an incentive to vote against it in order to gain opportunity to vote in favor of an even more preferred provision/tax package. With respect to valuation of an environmental project, Richer (1995) shows that his CV WTP estimates are influenced by information about whether a different alternative plan for a national park in California’s Mojave Desert was likely to be put forth if the current plan described in the survey was not approved. Another variant is where there is another party ( e.g ., another government agency or private entity) who potentially19

can provide the good. 15 The general principle is that direct linkage between a decision on one issue and a decision on another issue can cause difficulty in interpreting the result, as the optimal response of the agent should generally take the sequence of decisions and options into account. There is a further condition that is important for the interpretation of the results but not for the incentive properties of the RARAS mechanism. The agent needs to believe that if the agency implements a particular alternative: the specified good, Q, will be provided and the stated price, P, assessed. If instead the agent believes that Q* will be delivered and P* paid if this alternative is chosen by the agency, then the agent’s optimal response should be based upon (Q*, P*) not the stated (Q, P). Note this condition holds for interpreting actual votes or actual consumer purchases as well as responses to preference survey questions. 16 An important implication of this condition though is if the goods and prices used in a preference survey go

15 This problem appears to have influenced the Cummings et al. (1997) results. In that experiment, agents are randomly assigned to a “hypothetical” treatment and a “real” treatment in which the group votes on whether to contribute a specified amount per agent to provide the good. The estimate based upon the hypothetical treatment is higher than that of the real treatment, although Haab, Huang, and Whitehead (1999) show judgment of the significance of the difference depends upon how the larger variance in the “hypothetical” treatment is taken into account. We believe that to many of the agents interviewed in Georgia, the Cummings et al . hypothetical treatment should have appeared as an attempt to determine whether it was possible to mount a fundraising effort to provide printed information booklets on toxic hazards to poor people in New Mexico. As such, we would have expected the “hypothetical” treatment WTP to be higher than true WTP. However, uncertainty about why agents in Georgia should be asked about voluntary contributions to a New Mexico program may have lead to the larger variance found by Haab, Huang, and Whitehead (forthcoming). For the “real” treatment we would have expected an under-estimate of true WTP due to the possibility of having some other group pay to distribute the already printed booklets. A later experiment by Cummings and Osborne-Taylor (1998) effectively replicates this experiment but with additional treatments where there are different probabilities that the vote taken by the group is binding. The WTP estimate decreases from the “hypothetical” treatment to the “real” treatment as the probability that the group vote is binding goes from 0 to 1. This is the result that our model predicts if all treatments were perceived by agents as being consequential and that there are competing incentives to over pledge and free ride in all of the probabilistic treatments. The result that would be predicted theoretically if there was no incentive to over pledge in the “hypothetical” treatment and free ride in the “real” treatment would be that all of the treatments with a positive probability of the vote being binding should result in similar WTP estimates. 16 Carson et al. (1994) show, for instance, in a recent CV study in California that respondents who do not currently pay taxes are willing to pay more than respondents with otherwise identical characteristics. That respondents who believe that the state government would assess the one time tax in multiple years are willing to pay less than respondents who think the fee will only be applied one time and that respondents who don’t think that the plan will work completely are willing to pay less than those who think that it will work. See Randall (1994) for a discussion of this issue in the context of the travel cost model. There are large literatures in marketing and political science dealing with what are effectively the P’s and Q’s perceived by agents when they make decisions.20

beyond what the agent finds plausible, the preference survey question is likely to be answered on the basis of the expected good and the expected price rather than the stated ones.

Introduction of Cost Uncertainty Binary discrete choice preference surveys often provide a cost (in monetary or other terms) for each alternative and this cost information plays a key role in estimating welfare measures. What role should agent uncertainty over cost play in the answers given? The answer is obvious if the survey provides a cost estimate of $X and the agent thinks that since the government has a proclivity for cost overruns that the actual cost will be double the stated cost. The analysis should be performed with the cost as perceived by the agent. The more interesting case is when the agent takes the survey and provides $X as the expected value with some type of distribution around $X. Here the key issues can be seen to revolve around whether the original status quo choice set will still be available and whether a commitment to the pay for the good is required ex ante before the cost uncertainty is resolved. These two conditions determine whether shifts from an original “yes” to a “no” and vice versa are possible given a mean preserving increase in cost uncertainty. Table 2 displays the possible

outcomes.

Table 2: Effect of Increased Cost Uncertainty upon Binary Choice

Ex ante choice (i.e., commitment) Ex post choice (i.e., no commitment) Status Quo still available Can only shift Yes ® No Can only shift No ® Yes Status Quo no long available Can only shift Yes ® No Can shift either Yes ® No or No ® YesFor the case of provision of a public good with a coercive payment mechanism, the

status quo choice set is still available but one has to commit ex ante to paying the uncertain cost. This commitment translates into income uncertainty and hence is never preferred by risk adverse agents. Hence one would expect to see some shifts from “yes” to “no” responses. There should be no shifts in the opposite direction, so that the aggregate change is a decline in standard

statistics of the WTP distribution like the mean and median relative to the case with no cost uncertainty. The other case where an ex ante commitment is required has the same result but may be of less practical relevance since most examples here require an ex ante commitment to purchase a fixed quantity of the alternative to the status quo before the actual cost of the

alternative was observed. The opposite phenomena, possible shifts from original an “no” to “yes” response with increases in cost uncertainty, should occur in the case where the choice can be made ex post after the cost is observed and the status quo choice set is always still available. The main examples of this case are provision of a pubic good via voluntary contributions and the introduction of a new private good. The basic logic in this case is that since the status quo choice set will still be available, all agents will either favor or be indifferent the addition of the new alternative. Increasing the level uncertainty can cause some agents who were indifferent to the addition of the alternative to the choice set to favor it. Changes the a “yes” to a “no” response cannot occur, even though it is possible that an increase in cost uncertainty can make some agents, who were already in favor, worse off. The last of the four cases occurs where the only ex post commitment is required and the original status quo choice set will no longer exist, if the alternative to the status quo is provided. The main examples here are quasi-public goods and private goods where only one of two22

possible configurations of the good will be offered ( e.g., a low quality-low price recreation site could be transformed into is a high quality-higher priced version of the site) In this case, it obviously possible for increasing the degree of cost uncertainty to result in both shifts from “yes” to “no” and “no” to “yes.” There are a number of other informational issues that we do not expl ore here except to note that a formal analysis of the role of different types of uncertainty is likely to be more productive than the all too frequently invoked vague concept of agent unfamiliarity with a good as a justification for all types of apparent aberrant behavior. Much of the richness of economic theory in recent years has come from the introduction of different types of uncertainty and asking how agents should optimize in the face of it (Varian, 1992). Particularly, relevant here is the rapidly developing literature on how agents process information in candidate and referendum elections ( e.g., Popkins, 1991; Lupia, 1994). This literature suggests ways in which agents make reasonably informed decisions based on imperfect information. Further, simply providing more information does not necessarily lead agents to make the decisions closer to that which they would make if fully informed (Lohmann, 1994). 17 This suggests that the informational content of a survey used for environmental valuation should be examined to see if agents were given a reasonably complete, comprehensible, and balance presentation of the alternatives offered.

17 For example, consider an agent who initially favored a project and saw both its benefits and costs as being small. The agent, if fully informed, would still favor the project but realized that both its benefits and its costs were large. The agent informed that the cost of the project is large, but not given the corresponding benefit information, will now oppose the project. Much advertising in marketing and political campaigns operates on this notion of providing selective “half-truths.”23

Double-bounded Discrete Choice Questions The inherent problem with a binary discrete choice question is the limited information the response to it provides about the agent’s preferences. 18 Double-bounded discrete choice estimators have become popular in the environmental valuation literature because they tend to dramatically shrink the confidence intervals around point estimates of statistics of the willingness to pay distribution. The approach is straightforward. If the agent said “yes” to the initial cost amount asked, then ask the same question at a pre-chosen higher amount, and if the agent said “no” to the initial amount, ask the same question at a lower amount. 19 The initial presentations of the double-bounded format relied on double sampling/interval censoring statistical models (Carson, 1985; Carson and Steinberg, 1990; Hanemann, Loomis, and Kanninen, 1991). They assumed that agents have a single latent WTP value and that the responses to both the first and the second questions are based upon simply comparing this latent WTP value to the cost amount asked about in each question. Statistically, the implication of this assumption is that, with appropriate conditioning, there is perfect correlation between the WTP distributions implied by the responses to the two questions.

18 The only information provided is whether the agent’s WTP for the good is higher or lower than the single amount asked about in the survey question. It is possible to use parametric assumptions about the underlying WTP distribution to effectively overcome this sparse information, but such assumptions can play a large role in the estimates derived. Non-parametric approaches to the use of binary discrete choice data ( e.g ., Kriström, 1990) exist that make the power of these assumptions abundantly clear. 19 In some respects, the double bound model is similar to the iterative bidding game approach used in the early CV literature (Randall, Ives, and Eastman, 1974) that was usually found to suffer from a phenomena known as starting point bias whereby the amount initially provided the agent influences the agent’s final WTP amount. There are some key differences though which make the two approaches fundamentally different. The initial cost amount in the iterative bidding game was never intended to reveal information about the goods actual cost and the iterative steps from that amount are usually quite small. In contrast, the statistical tools used to analyze data from both the binary discrete choice and the double bound discrete choice formats exploit the agent’s conditioning on the cost number explicitly provided and the interval formed by the first and second price is fairly large. Most good studies using a double-bounded format go to some effort to provide a rationale to the agent as to why the cost number used in the second question is different from that of the first. An interesting variation on the double-bounded format is a single binary discrete choice format with a follow-up open-ended question. Farmer and Randall (1996) analyze this formatfrom a theoretical and empirical perspective and show results similar to those described here for the double-doublebounded estimator: the second responses tend to be biased downward.24

Following Cameron and Quiggan’s (1994) pioneering examination of this assumption, several stylized facts have emerged concerning the comparison of the WTP estimates based on the first binary discrete choice question and both binary discrete questions: (a) the WTP distributions implied by the first and second questions are not perfectly correlated, (b) the WTP estimate based upon just the first estimate is higher than the WTP estimate based upon both questions, and (c) the number of negative responses to the second question is higher than would be expected based upon the WTP distribution estimate from the first question alone. Alberini, Kanninen, and Carson (1997) have put forth a general error-components model, and McLeod and Bergland (1999) have put forth a Bayesian preference-updating model to handle these issues. What sort of effects should the asking of a second binary discrete choice question have on the latent WTP distribution? The key property of this format from our perspective is that the agent has been told that the same Q was available at two different prices. The best-case scenario here is that the agent takes the second price as the expected price but now considers the price to have some uncertainty surrounding it. 20 Consistent with the discussion in the previous section, statistics such as mean or median WTP will be shifted downward in the second question for risk adverse agents and public goods even though preferences for it have not changed. There are, however, several other plausible alternatives for what the act of asking the second price should signal to agents. One of these is that the agency is willing, in some sense, to bargain over the price. For agents who originally answered “no” and got asked a lower price, the optimal response may be to answer “no” again in hopes of getting offered an even lower price. 21 This should result in the second WTP response being “no” for some of these agents even though

20 Alternatively, if the agent thought the first price had some uncertainty surrounding it, asking the second price should increase the original level of uncertainty since for the double-bounded estimator the first and second prices are typically fairly far apart.25

had this amount been asked at the first question the response would have been “yes.” A similar effect can be found with respect to those whose original answer was “yes”. Since the good was originally offered at a lower price, it can presumably be provided with some positive probability at the initial price. As such, some agents will find it in their self-interest to risk not getting the good by holding to the lower price and saying “no” to the second higher price, even though the agent’s WTP for the good exceeds the second price. The effect of this type of behavior would be to shift the WTP distribution implied by the second question to the left, and hence, reduce

estimates of mean and median WTP.

A nother plausible assumption is that the actual cost to an agent will be some type of weighted average between the two prices. If this assumption is made, the second question should be answered on the basis of this weighted average of the two prices. It is straightforward to see that for an initial “no” response, that any weighted average of the first and second prices is higher than the second price. For an initial “yes” response, any weighted average of the first and second prices is lower than the second price. 22 The last plausible assumption we consider is that the agent might interpret the signal given by the second price as implying that the quantity has changed to match the changed price in a consistent manner. For an initial “no” response, the shift in quantity that is consistent with the reduction in price is to reduce the perceived quantity/quality of the good that would be provided. The implication of this is to shift the WTP distribution implied by the second response to the right for these respondents. This is a commonly voiced concern in focus groups and debriefing questions. For agents who initially said “yes”, the shift in perceived quantity is

21 It is of course possible to expand the double-bounded concept to asking a third question. See Bateman et al. (1995) for an example.26

upward. There does not appear to be any collaborating evidence to support the proposition that this is a common phenomenon. What should be grasped from this discussion is that to a rational agent the second price must signal that something is going on. All of the plausible assumptions lead to the correlation between the WTP distributions implied by the two questions being less than 1. All of these assumptions also shift the WTP distribution implied by the second question to the right for agents who initially gave a “no” response, and hence, produce an “excess” number of no-no responses. For agents initially giving a “yes” response, it is possible for the WTP distribution implied by the second question to shift either to the left or the right. However, only the price averaging assumption has much credence in terms of the possibility of producing an upward shiftin the standard WTP statistics. On balance, we would expect that WTP estimates from a doublebounded format to be smaller than those from a single-bounded format. All of these hypothesestend to be strongly supported by the empirical evidence. It may be desirable to use the doublebounded format in CV studies; however, this desirability rests on the analyst’s tradeoff between the likely downward bias and the tighter confidence interval (Alberini, 1995).

Continuous Response Formats Ideally one would like to have the agent’s actual WTP or WTA, not a discrete indicator of it. So it is not surprising that many early CV studies used an open-ended direct question. 23 Many economists thought that these early efforts would fail because agents would give the

22 Note that this assumption is not inconsistent with the arguments concerning uncertainty and the two may be combined. For initial “no” responses, this effect of adding uncertainty is reinforcing in a downward direction. For initial “yes” responses, the effect is in the opposite direction and mitigates the upward effect of price averaging. 23 The continuous response format is known as a matching question in the psychology literature and is a special type of open-ended question in the survey research literature.27

extremely high WTP answers. This did not happen ( e.g ., Brookshire, Ives, and Schulze, 1976), and interest in the survey based valuation methods grew in part due to this anomaly. The early problem that researchers did find with the direct question was that agents always wanted to know what the project would cost them. Agents did not understand why they were not provided the cost information if the agency had worked out the details of how the good would be provided. Further, many agents appeared to have great difficulty formulating a(continuous) WTP response. This led to very high non-response rates and a large number of socalled “protest zeros” which were typically dropped from the analysis. This lead to speculation that survey respondents did not have “well-defined” preferences in an economic sense. Three different directions were tried to overcome this problem. The binary discrete choice format (Bishop and Heberlein, 1979) discussed earlier gets around one of the key problems by giving agents the cost number they want and then uses a statistical analysis that “appropriately” conditions on agents reacting (favor/not favor) to that cost number. The earlier iterative bidding game method suggested an initial amount and iterates up or down from that amount in small increments (Randall, Ives, and Eastman, 1974). The payment card approach asks agents to pick a number (or any number in between) on a card (Mitchell and Carson, 1986; Cameron and Huppert, 1991). The latter two methods can come close to achieving a WTP response in continuous terms; and, except when these formats have special properties, the discussion of the continuous response format will apply to these formats as well. With different elicitation formats came the inevitable urge to compare their results ( e.g., Smith and Desvousges, 1986). Researchers were dismayed to find that different response formats lead to different WTP estimates and the divergence between these estimates is frequently cited as one of the major reasons why estimates based on stated preference questions should be rejected28

(Hausman, 1993; McFadden, 1994). 24 The stylized fact here is that discrete choice formats produce higher WTP estimates than do continuous response formats ( e.g. , Boyle et al ., 1996). Should the divergence in estimate s from different formats be surprising? 25 No. Given the Gibbard-Satterthwaite result, it is impossible to formulate a continuous response question thathas the same incentive and informational properties as an incentive-compatible binary discretechoice question. Many researchers looking at the results, however, have been misled by the facevalue dilemma. The divergence between the estimates from the different formats suggested that either agents were not truthfully revealing their preferences to one or more of the elicitation formats or that they did not have well-defined preferences in the sense used by economists. As noted earlier in this discussion, the expectation of many economists was that most agents would provide very large WTP responses when asked an open-ended WTP question if agents were acting strategically but not truthfully revealing their preferences. However, the opposite phenomenon was observed: estimates from binary discrete-choice questions were higher than those from continuous response CV questions and continuous response CV questions contained lots of zero responses. Faced with an open-ended question, a very large WTP response does turn out to be the optimal strategy for an agent who believes (a) the cost of the public good to the agent is fixed, (b) her true willingness to pay for the good is larger than the cost if provided, and (c) the good is more likely to be supplied the larger the sum of the willingness to pay responses given by agents.

24 The irony in this position is that estimates of other economic quantities based upon substantially different econometric techniques have typically differed even though data on actual behavior was being used. The usually recommended approach in this situation has not been to discard economic theory and econometric methods but rather to understand the source of the differences.

25 From the critique by cognitive psychologists, the divergence between framing provided by the (binary discrete) choice and open-ended matching question is at the heart of problems with microeconomic theory (Tversky, Slovic, and Kahneman, 1990).29

Note that only the subset of agents whose WTP is greater than their cost should be giving a positive WTP response, so one should never have expected all agents to engage in this behavior. Condition (c) corresponds to the benefit-cost criteria, but it is hard to find a single instance where an agency decision has been made based purely on that criteria. There is little evidence to suggest that agents believe that the agency is simply summing their WTP responses. As such, we believe it useful to consider a variety of other beliefs that agents may hold. Let’s first consider the optimal response of an agent whose perceived cost of the public good is greater than the agent’s willingness to pay. Maintaining the previous assumptions, this agent’s optimal response is “zero”. This result turns out to be fairly robust to plausible alternatives to (c) that we discuss below and, as such, may help to explain the large number of zero responses received to open-ended type questions. The intuition behind this result is that the agent’s utility is reduced if the public good is provided and the cost assessed against the agent.The response that adds the least amount to the sum of the benefits (given the usual nonnegativity constraint in the open-ended format) is “zero.” Step back for a moment from the benefit-cost cr iteria that has dominated economic thinking on the incentive structure of the open-ended question and recognize that the simple act of asking an open-ended question is likely to signal to agents that the cost allocation among agents for providing the good is not fixed. Once the agency is prepared to shift the vector of costs facing agents, changing condition (a) above, toward increasing the cost to agents having (relatively) high WTP for the good and decreasing it to those who do not, the incentives for agents whose WTP is greater than the initially perceived cost change substantially. These agents now have to balance the increased probability that the good will be supplied with a high WTP30

response against the potential upward shift in the cost they will pay if the good is provided. For agents having WTP less than the initially perceived cost, the optimal response is still zero. Since the government rarely if ever uses a pure benefit-cost criteria, it may be plausible for agents to assume that the agency is simply trying to determine what percentage of the relevant population has a WTP higher than the cost which may or may not be assumed to be known to the agency at the time of the survey. Combined with the potential to reallocate the cost burden, the optimal response of an agent whose WTP is greater than the initially perceived cost is now equal to the cost while the optimal response of an agent whose WTP is less than the initially perceived cost the optimal response is still zero. In all of these cases, the optimal response depends strongly on the agent’s perception of the agency’s cost of providing the good. The agent should first compare her actual WTP to the expected cost. The optimal response for agents whose WTP is less than the perceived cost, under most plausible uses of the information provide, is zero. Such an agent should additionally “protest” in any other way possible, as the change from the status quo will negatively impact the agent’s utility. The optimal response for an agent whose actual WTP is greater than expected cost depends upon her belief about how the agency will use the stated WTP. In this case, the optimal response typically is further conditioned on expected cost. The difficulty in interpreting the positive WTP response is that different agents may have different beliefs about agency use of the response. Agents, however, don’t know the cost with certainty. They can formulate priors about the cost and should incorporate any information provided in the survey that they believe is related to cost. This type of behavior would give rise to starting point bias in iterative bidding games (Boyle, Bishop, and Welsh, 1985) and range/placement effects in studies using payments cards31

(Rowe, Schulze, and Breffle, 1996) to the extent that agents think that the “extra” information provided in these formats is correlated with cost.

On occasion, a variety of different open-ended formats that are said to be incentive compatible are used in a survey context such as the Becker-DeGroot-Marshack mechanism or the Vickery auction. 26 Both of these mechanisms elicit a continuous WTP response. There aretwo things to remember about such mechanisms. First, they do not get around the GibbardSatterthwaite result. Holt (1986) and Karni and Safra (1987) (hereafter HKS) independently showed such mechanisms depend crucially on the preferences obeying the expected utility assumption. Many researchers are willing to maintain the expected utility assumption, and many key economic results on risk are locally robust to most non-expected utility alternatives (Machina, 1995). However, when trying to implement either of these two mechanisms in a survey context, the difficulty lies much deeper. Both of these mechanisms rely on the ability to condition the agent’s response on an “exogenous” random element. We have shown that it is impossible to formulate a simple open-ended matching question that is informationally and strategically equivalent to an incentive compatible binary discrete choice question in a survey context. This result is a companion of the HKS theorem. To make the matching question equivalent strategically to the binary discrete choice, the agency has to pre-commit either to the cost or to an exogenous device to provide the cost. Doing so prevents the agency from exploiting the extra information that the agent provides in the matching format but not in the choice format. To get the agent to reveal the matching answer, the agent cannot know the cost. The need for the agent’s uncertainty about the cost puts one back in the HKS world where expected utility is

26 Other mechanisms eliciting a continuous response like the Groves mechanism (Groves, 1973) require stronger restrictions on preferences ( e.g ., quasi-linearity in income) and the possibility of side payments.32

required. The need for agency pre-commitment not to exploit the extra information contained in the continuous WTP response effectively prevents its being used in a survey context.

Sequence of Paired Comparisons In addition to wanting tighter confidence intervals on the WTP distribution for a single good, decision makers often want information on the WTP distributions for a variety of related but different goods so that they can pick the best option. There are two popular approaches in the literature for doing this. The first is to offer agents a sequence of paired comparisons. The second is to ask agents to pick between or rank order a set of k > 2 alternatives. Here we discuss the strategic issues that arise with these choice formats and do not deal with issues related to the adequacy of the information set on each distinct good provided in the survey. In an ideal world in which the objective involves valuing public goods, the agent treats each paired comparison independently and the desirable properties of a single binary discrete choice question with a coercive payment requirement can be repeatedly invoked. There is a very simple question, however, that illustrates the fundamental difficulty with a sequence of paired comparisons. Consider the case of air pollution levels in a city. The agent is asked to pick between different pairs of air pollution levels that involve different costs and different health effects and visibility levels. Since air pollution in the city is a public good, however, all agents will eventually face the same air pollution level. If k different air pollution levels are described to the agent in the course of the sequence of paired comparison, the agency must have some method of choosing among the k different levels. Any particular method that the agent perceives that the agency is using to incorporate agent preferences into its choice of an air pollution level generally will provides incentive for non-truthful preference revelation. In some instances, it will even be33

optimal for the agent to reject his or her most preferred level (out of the k) in a particular paired comparison. Once this is possible, the standard methods of inferring value from choices no longer work. The essential problem is that an agent’s optimal choice depends both upon the agent’s preferences, expectations about what the other agents will do, and the perceived rule for aggregating the results of each paired comparison. This result has long been established in the literature on the properties of voting rules (Moulin, 1994). With quasi-public and private goods, the difficulties noted for public goods still exist, with the exception that it may be possible for more than one of the k goods to be provided. This possibility tends to reduce the likelihood that an agent will make a choice that is not her favorite; and in the next section on multinomial choice, we discuss the aggregation issue further. 27

Multinomial Choice Questions Many of the issues raised in the previous section on a sequence of paired comparisons are relevant to multinomial choice questions. The strategic issue that an agent faces when answering a multinomial choice question (pick the most preferred out of k > 2 alternatives) is how the agency translates the responses into actions. The simplest case consists of generalizing the decision rule used in the binary discrete choice format by assuming that the agency will provide only one of the k goods and that the higher the percentage of the sample picking any particular alternative, the more likely that alternative will be provided. The well-known result from the

27 There are further issues related to a sequence of paired comparisons that need to be addressed in any particular analysis. The first of these is the strong possibility that the scale term associated with each paired comparison is different. If this is the case then much of the gain in precision and the ability to deal with changes in attributes associated with asking the sequence of paired comparisons may be an illusion. The second is that most rules for combining information from different paired comparisons implicitly require that the independence of irrelevant alternatives assumption to hold. This property is routinely rejected in paired comparison data. The third involves the common use of pairs where both alternatives are off the agent’s current utility frontier and neither represents the status quo. This practice requires much stronger assumptions about the nature of the agent’s utility function than typically assumed in order to combine the data from different paired comparisons.34

voting literature on multi-candidate races with a simple plurality winner is that the race from an agent’s strategic perspective reduces to a binary choice between the two alternatives that the agent believes will receive the largest votes independent of the agent’s vote. The rationale behind this result is straightforward: only the top two alternatives have a chance of winning; picking the most preferred alternative among these two will maximize the utility of the agent’s final

outcome. 28 The agent is truthfully revealing her preferences, but such truthful preference revelation is, as it should be, conditional on the expectations about the choices of the other agents. However, the agent is not answering the question of interest to the analyst. It will be optimal in many instances for the agent to pick an alternative other than the (unconditionally) most preferred one. Let us now consider perhaps the oppo site case, one of particular relevance to private and quasi-public goods, by changing one of the key assumptions. Now instead of only one of the k goods being supplied, let k-1 of the goods be supplied. To keep matters simple, assume further that the agent only uses at most one of the goods. Examples of such a choice context might be a government agency that had to close four out of five recreational fishing lakes or a computer company that was going to offer four out of five configurations of a particular computer model. In this case, it is optimal for the agent to pick the most preferred alternative out of those offered. Formally, it can be shown that this case collapses to a binary discrete choice of the agent’s most preferred alternative against another stochastically chosen alternative. To see this, note that the worst possible outcome for the respondent is that the agent’s first choice is not made available.

28 With a richer model of agent expectations, it may be optimal for the agent to vote for an alternative that is not one of the top two if there is enough uncertainty over the expected finish of alternatives and the utility differences between the alternatives is large enough. The manifestation of this proposition can be seen in the behavior of fringe political candidates in plurality winner elections. Such candidates try to convince voters that they have a non-trivial chance of winning, that the difference in positions between the two front-runners is extremely small, and that they are much closer to the voter’s ideal point.35

Because all of the other alternatives are provided, the agent’s second choice will be available. Effectively, this is a determination of what alternative will not be provided and in pairing the agent’s favorite alternative against any of the other alternatives, the agent’s optimal response is to pick her most preferred. The general result we have sh own is that, if all but j of the alternatives are to be provided, then the alternative chosen by the agent should be one of the agent’s j favorites. Often the number of alternatives that will be provided is unknown to the agent at the time of making the multinomial choice. A stochastic version of this result has the agent trading off the utility of sets of alternatives with different maximum elements against the agent’s prior on j and the agent’s priors on the choices made by the other agents. Doing so reveals that agents will pick either their (unconditionally without considering the responses of other agents) favorite alternative or close to it, as long as one of three conditions holds: the expectation of j is fairly small, the utility difference between the agent’s most favorite alternatives and the other alternatives is large, or the prior on the choices by the other agents is fairly uninformative. The implication of this is that agents will appear to make mistakes or optimization errors more often. If they don’t pick their favorite, they should pick an alternative close to it. The statistical manifestation of this type of behavior is a violation of the error term properties associated with the independence of irrelevant alternatives (IIA) assumption. In empirical applications of this elicitation format, the IIA assumption is usually violated. While there are a number of other good reasons for this assumption being violated, such as the rationale behind classic red bus-blue bus problem, it is usually impossible to separately identify the reason for an IIA violation. From a purely statistical viewpoint, it is possible to deal with the problem by introducing one or more scale/variance terms (Swait and Louviere, 1993). This is often36

sufficient for looking at marginal tradeoffs between attributes. To uniquely recover the latent WTP distribution, it is necessary to have an estimate of the correct scale factor. 29 The optimal strategic behavior in this case is often observationally equivalent to direct manipulation of the scale parameter and making recovery of the correct scale factor impossible. 30 Tests for whether data from stated preference surveys and revealed preference observations are consistent with each other and can be combined after (potentially) allowing for a difference in the scale factor (Adamowicz, Louviere and Williams, 1994) are tests against random responses in the stated preference data, not tests against strategic behavior.

With either subadditivity or superadditivity of the utility of the different alternatives and k-j ( j > 1) alternatives to be provided, it is possible to find conditions where the agent should indicate her unconditionally least preferred alternative. The rationale here is that the agent’s outcome utility is defined on the set of goods to be provided, not the individual goods taken independently. This is a hopeless situation for learning anything reliable about agent preferences for individual goods.

An alternative to asking agents to pick their single most preferred alternative out of k , is

to ask them to rank order all k alternatives. This exercise could potentially provide considerably more information, but an analysis of the agent’s strategic incentives becomes considerably more difficult. The same issue for the agent still exists: how does the agency translate the ranks into a

choice of which of the k alternatives to provide. Methods for dealing with rank data in a manner

29 This correct scale factor can often be obtained in studies involving quasi-public or private goods from a model estimated on the more limited set of choices current available in the market. This will not typically be the case for public goods. 30 The scale parameter is typically the negative inverse of the price coefficient. The agent’s optimal strategy is to induce the agency to supply the good with the most desired set of attributes at the lowest price. The simple way to do this is to pick the favorite anytime the price is low and otherwise pick something close to it with a low price. Formulated in terms of the expected minimum cost that the agent believes the agency would provide the good at, the agent wants to appear to have an infinite demand elasticity at this cost and to be uninterested above that cost.37

consistent with economic theory effectively require the IIA assumption to hold for all possible subsets of the ranked data. This implies that it is possible to explode the data to form sets of multinomial choice questions (Chapman and Staelin, 1982). The IIA assumption can be tested but it does not appear to generally hold for contingent ranking data, and welfare estimates can be substantially impacted if the IIA assumption does not hold ( e.g., Hausman and Ruud, 1987). 31

Concluding Remarks

We have argued that serious consideration should be paid to the incentive and informational properties of preference questions. Much of the difficulty with interpreting the apparent anomalies 32 associated with the estimates based on preference survey questions revolves around what we call the face-value dilemma: either agents always truthfully reveal their preferences to survey question as stated or they do not. This is a false dilemma. Simple common sense economic models predict large divergences between what agents say they will voluntarily contribute to provide a public good and what they actually contribute. There are now many studies that demonstrate this prediction empirically. The difficulty lies not in the theory or the experimental demonstration but rather in the interpretation that is often placed on these results. Rather than be taken as evidence that respondents don’t have well-

31 A major problem occurs when there are a group of respondents who do not appear to want to trade-off one of the attributes against money. The appearance of such lexicographic preferences can lead to infinite WTP estimates. A subtler problem occurs in that the variance of the error term appears to be substantially larger for “middle” ranks than the most and least preferred alternatives. 32 The term anomaly is often loosely used. It is possible to have results that represent anomalous behavior from the perspective of economic theory and it is also possible to have such behavior occur in a survey. The most interesting anomalies from the perspective of this paper are those that only occur in surveys. The first step to take with such an anomaly is to see if it can be observed in settings not involving surveys. A number of anomalies first alleged to be survey specific have been shown to be easily replicable in experimental contexts and examples readily identifiable in common market transactions. These include preference reversals (Grether and Plott, 1979), large divergences between WTP and WTA (Bishop and Heberlein, 1990), and part-whole bias (Bateman et al ., 1997). In some of these38

defined preferences, differences between the estimates obtained using different elicitation formats, if predicted by economic theory, should be taken as evidence supporting the proposition that respondents are taking the scenario posed seriously. Divergences between binary discrete choice and double-bounded formats or between binary discrete choice and open-ended formats are likewise consistent with theory. Optimal response strategies in most cases are fairly simple, and in many instances, such as the zero responses to open-ended type questions, fairly robust to alternative assumptions made about agent beliefs. In some situations, particular elicitation formats should be avoided altogether,

while in others one faces a classic bias versus variance trade-off. The researcher should understand the trade-off being made in the choice of an elicitation format. Claims about the specific incentive and informational properties of a particular elicitation format should not be made in the abstract. Careful attention needs be paid to the type of good being offered, the nature of the payment obligation for the good, and other aspects of the context in which the good is offered in order to clearly determine incentive and informational properties. For the binary discrete choice format, the introduction of a new private good turns out to be one of the worst cases for truthful preference revelation. The other bad case is to compare survey indications of willingness to voluntary contribute to provide a public good to actual contributions. Here neither estimate should approximate the true underlying WTP. One need not cast a binary discrete choice question explicitly as a vote in a referendum to get an incentive compatible question; it is sufficient to structure the question as advice to the government on the issue, a result that should be of use to researchers in areas where referenda are not frequently

held.

instances, such as the often-noted WTP-WTA divergence, models predicting such divergences consistent with standard neoclassical economic theory have been proposed (Hanemann, 1990).39

None of our analysis has relied on agent experience or familiarity with the good. While these may influence the agent’s true WTP for the good, they do not influence the incentive properties of question format in the context in which it is being used. Nor have we relied on any notion that agents learn about preferences and update them. Informational and incentive properties of formats do play a role in updating of optimal response strategies. Indeed, it is possible to recast some Bayesian models, such as the recent work of McLeod and Bergland (1999), as Bayesian updating not with respect to preferences, but rather, with respect to determining the optimal strategic response. A number of elicitation formats commonly used in marketing research are currently attracting considerable attention in environmental valuation, both for the hope that more information can be collected from each agent (than can be collected with the binary discrete choice format) and for the hope that these newer formats will have fewer problems than does a binary discrete choice format. From an incentive perspective, this latter hope is likely to be misplaced with respect to the two most common valuation situations in environmental economics, the provision of public goods provided by the government and changes in a single quasi-public good provided by the government. The generalization of the binary discrete choice format in the directions used by marketing researchers causes it to lose its desirable incentive properties. Further, as the number of goods that must be described in a survey increases the time available to describe each good shrinks. For the introduction of new private goods, the multinomial choice format may be close to incentive compatible from the perspective of estimating marginal trade-offs between attributes as long as the perceived number of goods that are likely to be provided is sufficiently large. This is because deviations from truthful preference revelation are most likely to impact the scale parameter that drops out of marginal comparisons.40

This fortunate occurrence is less likely to be true for estimating the total value of a good since that calculation requires a consistent estimate of the true scale parameter. Our work suggests that there are different natural underlying economic structures to different valuation problems. The typical problem in environmental valuation is the determination of the total value of a single good or a non-marginal change in a single good. The strategic incentives facing agents confronting this problem may be the major force that has moved researchers away from the open-ended and ranking formats toward binary discrete choice formats. The typical problem in marketing is the determination of a trade-off between different attributes of goods when many competing goods will be offered. Researchers in this area have moved from open-ended and discrete choice questions, to ranking questions, and then to the

multinomial choice format. A shift to the paired comparison or multinomial choice format has sometimes been recommended as a means of reducing or eliminating the sensitivity of the estimate of the value of a particular good to the sequence in which it was valued. However, this sensitivity is not a problem of elicitation format. Attempts to get sequence effects to go away by shifting to a question format that explicitly involves multiple goods are misguided. One of the major differences between private goods and public goods is that, for the former, agents themselves largely determine the order in which they obtain information about goods and make purchases of them. For public goods, the government through its control of the agenda determines the order in which projects are considered. Sequence effects (Carson, Flores, and Hanemann, 1998) are inherent to sequential decision making and because substitution effects enter WTP calculations much differently than they do demand calculations, sequence effects are apt to be large. Flores41

(1994) shows that the classic agenda control problem can be rewritten in terms of WTP and WTA sequences. In closing, a remark on the term hypothetical , frequently affixed as an adjective in front of the word survey, is in order. In a famous and often cited remark on the early use of surveys for environmental valuation, Scott (1965) bluntly states: “Ask a hypothetical question and you get a hypothetical answer.” Hypothetical as used here seems to imply that the responses are to an “imaginary” inconsequential situation, and, as such, the responses will have no influence on any relevant decision. From an economic perspective nothing can be inferred about respondent preferences from asking such a question.

The term hypothetical , however, also means conjecture, counterfactual, and contingent. This is the context usually used by researchers who ask preference questions. It is consistent with

our definition of a consequential survey but an incomplete one because we require the agent to care about the alternatives and the agent to perceive that the agency will take the survey responses into account in its decision making. Our suggestion is to eschew the use of the word hypothetical

in discussing preference questions in favor of consequential and inconsequential

to emphasize the conditions requisite for the application of economic theory.42

References Adamowicz, W. J. Louviere, and M. Williams (1994), “Combining Revealed and Stated Preference Measures for Valuing Environmental Amenities,” Journal of Environmental Economics and Management , 26, 271-292. Adamowicz, W., J. Swait, P. Boxall, and J. Louviere (1997), “Perceptions Versus Objective Measures of Environmental Quality in Combined Revealed and Stated Preference Models of Environmental Valuation,” Journal of Environmental Economics and Management , 32, 65- 84.Alberini, A. (1995), “Efficiency vs Bias of Willingness-to-Pay Estimates: Bivariate and IntervalData Models,” Journal of Environmental Economics and Management , 29, 169-180. Alberini, A., B. Kanninen, R.T. Carson (1997), “Modeling Response Incentive Effects in Dichotomous Choice Contingent Valuation Data,” Land Economics 73, 309-324. Arrow, K., R. Solow, P.R. Portney, E.E. Leamer, R. Radner, and H. Schuman (1993), “Report of the NOAA Panel on Contingent Valuation.” Federal Register 58, 4601-4614. Bateman, I.J., I.H. Langford, R.K. Turner, and K.G. Willis (1995), Elicitation and Truncation Effects in Contingent Valuation Studies,” Ecological Economics , 12, 161-179. Bateman, I.J., A. Munro, B. Rhodes, C. Starmer, and R. Sugden (1997) “Does Part-Whole Bias Exist?: An Experimental Investigation,” Economic Journal , 107, 322-332. Bishop, R.C. and T.A. Heberlein (1979), “Measuring Values of Extra-Market Goods: Are Indirect Measures Biased," American Journal of Agricultural Economics , 61, 926-930. Bishop, R.C. and T.A. Heberlein (1990), “The Contingent Valuation Method,” in Economic Valuation of Natural Resources: Issues, Theory, and Applications , R.L. Johnson and G.V. Johnson, eds. (Boulder, CO: Westview Press). Bergstrom, T., L. Blume, and H. Varian (1986), “On the Private Provision of Public Goods,” Journal of Public Economics , 29, 2-49. Boxall, P.C., W.L. Adamowicz, J. Swait, M. Williams, and J. Louviere (1996), “A Comparison of Stated Preference Methods for Environmental Valuation,” Ecological Economics , 18, 243- 253. Boyle, K.J., R.C. Bishop and M.P. Welsh (1985), “Starting Point Bias in Contingent Valuation Bidding Games, “ Land Economics , 61, 188-194. Boyle, K.J, F.R. Johnson, D. McCollum, and W.H. Desvousges, R.W. Dunford, and S.P. Hudson (1996), “Valuing Public Goods: Discrete Versus Continuous Contingent-Valuation Responses,” Land Economics , 72, 381-396. Brookshire, D.S., B.C. Ives, and W.D. Schulze (1976), “The Valuation of Aesthetic Preferences,” Journal of Environmental Economics and Management , 3, 325-346. Cameron, T.A., (1988), “A New Paradigm for Valuing Non-Market Goods Using Referendum Data: Maximum Likelihood Estimation by Censored Logistic Regression,” Journal of Environmental Economics and Management , 15, 355-379.43

Cameron, T.A. and D.D. Huppert (1991), “OLS versus ML Estimation of Non-Market Resources Values with Payment Card Interval Data,” Journal of Environmental Economics and Management, 17, 230-246. Cameron, T.A. and J. Quiggin (1994), “Estimation Using Contingent Valuation Data from a "Dichotomous Choice with Follow-up" Questionnaire,” Journal of Environmental Economics and Management , 27, 218-234. Carson, R.T. (1985), "Three Essays on Contingent Valuation," unpublished dissertation, University of California, Berkeley. Carson, R.T., N.E. Flores, K.M. Martin and J.L. Wright (1996), "Contingent Valuation and Revealed Preference Methodologies: Comparing the Estimates for Quasi-Public Goods,” Land Economics , 72, 80-99. Carson, R.T., T. Groves, and M.J. Machina (1997), “Stated Preference Questions: Context and Optimal Response,” paper presented at the National Science Foundation Preference Elicitation Symposium, University of California, Berkeley. Carson, R.T., N.E. Flores and W.M. Hanemann (1998), “Sequencing and Valuing Public Goods,” Journal of Environmental Economics and Management, 36, 314-323. Carson, R.T., W.M. Hanemann and R.C. Mitchell (1987). “The Use of Simulated Political Markets to Value Public Goods.” Discussion paper 87-7, Department of Economics, University of California, San Diego, March. Carson, R.T., W. M. Hanemann, R.J. Kopp, J.A. Krosnick, R.C. Mitchell, S. Presser, P.A. Ruud, and V.K. Smith (1994), "Prospective Interim Lost Use Value Due to DDT and PCB Contamination in the Southern California Bight," report to National Oceanic and Atmospheric Administration, September. Carson, R.T. and D. Steinberg (1990), "Experimental Design for Discrete Choice Voter Preference Surveys," in 1989 Proceeding of the Survey Methodology Section of the American Statistical Association , (Washington: American Statistical Association, 1990). Champ, P.A., R.C. Bishop, T.C. Brown, and D.W. McCollum (1997), “Using Donation Mechanism To Value Nonuse Benefits From Public Goods,” Journal of Environmental Economics and Management , 33, 151-162. Chapman, R.G. nd R. Staelin (1982), “Exploiting Rank Ordered Choice Set Data Within the Stochastic Utility Model,” Journal of Marketing Research , 19, 281-299. Cronin, F.J. (1982), “Valuing Nonmarket Goods Through Contingent Markets,” report to U.S. Environmental Protection Agency by Battelle Memorial Institute, Richmond Washington. Cummings, R.G., G.W. Harrison and E.E. Rutström (1995), “Homegrown Values and Hypothetical Surveys: Is the Dichotomous Choice Approach Incentive Compatible?", American Economic Review 85, 260-266. Cummings, R.G., S. Elliott, G.W. Harrison and J. Murphy (1997), “Are Hypothetical Referenda Incentive Compatible?,” Journal of Political Economy 105, 609-621.44

Cummings, R.G. and L. Osborne. (1998), “Does Realism Matter in Contingent Valuation Surveys?” Land Economics , 74, 203-215. Farmer, M.C. and A. Randall (1996), “Referendum Voting Strategies and Implications for Follow-Up Open-Ended Responses,” paper presented at the annual U.S.D.A. W- 133 meeting, Jekyll Island, GA, February. Farquharson, R. (1969), Theory of Voting. (New Haven: Yale University Press). Flores, N.E. (1994), "The Importance of Agenda and Willingness to Pay." Paper presented at the 1994 Public Choice Society Meetings, Austin, TX, April. Gibbard, A. (1973), “Manipulation of Voting Schemes: A General Result,” Econometrica 41, 587-601. Green, J.R. and J.J. Laffont (1978), “A Sampling Approach to the Free Rider Problem,” in Agnar Sandmo, ed., Essays in Public Economics (Lexington, MA: Lexington Books). Greenhalgh, C. (1986), “Research for New Product Development,” in Consumer Market Research Handbook, 3 rd

ed., R.M. Worcester and J. Downham, eds. (Amsterdam: NorthHolland). Grether, D.M and C.R. Plott (1979), “Economic Theory of Choice and the Preference Reversal Phenomenon,” American Economic Review , 69, 623-638. Groves, T. (1973), “Incentives in Teams,” Econometrica, 41, 617- 631. Groves, T., R. Radner, and S. Reiter, eds. (1987), Information, Incentives, and Economic Mechanisms: Essays in Honor of Leonid Hurwicz (Minneapolis: University of Minnesota Press). Haab, T.C. and K.E. McConnell (1998), “Referendum Models and Economic Values: Theoretical, Intuitive, and Practical Bounds on Willingness to Pay,” Land Economics , 74, 216-229. Haab, T.C. and K.E. McConnell (1997), “Referendum Models and Negative Willingness to Pay: Alternative Solutions,” Journal of Environmental Economics and Management , 32, 251-270. Haab, T.C., and J.C. Huang, and J.C. Whitehead (1999). “Are Hypothetical Referenda Incentive Compatible?: A Comment,” Journal of Political Economy , 107, 186-196. Hanemann, W.M. (1984a), “Discrete/Continuous Models of Consumer Demand,” Econometrica , 52, 541-561. Hanemann, W.M. (1984b), “Welfare Evaluations in Contingent Valuation: Experiments with Discrete Responses,” American Journal of Agricultural Economics , 66, 335-379. Hanemann, W.M., and B. Kanninen (1999), "The Statistical Analysis of Discrete-Response." In Valuing the Environment Preferences: Theory and Practice of the Contingent Valuation Method in the US, EC and Developing Countries , edited by Ian Bateman and Ken Willis (Oxford: Oxford University).45

Hanemann, W.M., J. Loomis, and B. Kanninen (1991), “Statistical Efficiency of Double Bounded Dichotomous Choice Contingent Valuation,” American Journal of Agricultural Economics, 73, 1255-1263. Hausman, J.A., ed. (1993), Contingent Valuation: A Critical Assessment (Amsterdam: NorthHolland).Hausman, J.A. and P. Ruud (1987), “Specifying and Testing Econometric Models for RankOrdered Data,” Journal of Econometrics , 34, 83-104. Hensher, D.A. (1994), “Stated Preference Analysis of Travel Choice—The State of Practice,” Transportation , 21, 107-133. Hoehn, J. and A. Randall (1987), “A Satisfactory Benefit Cost Indicator from Contingent Valuation,” Journal of Environmental Economics and Management 14, 226-247. Holt, Charles A. (1986), “Preference Reversals and the Independence Axiom,” American Economic Review , 76, 508-515. Hurwicz, L. (1986), “Incentive Aspects of Decentralization,” in Handbook of Mathematical Economics, vol. III, K.J. Arrow and M.D. Intrilligator, eds. (Amsterdam: North-Holland). Inforsino, W.J. (1986), “Forecasting New Product Sales From Likelihood of Purchase Ratings (with discussion),” Marketing Science , 5, 372-390. Johannesson, M., B. Liljas, P.O. Johansson (1998), “An Experimental Comparison of Dichotomous Choice Contingent Valuation Questions and Real Purchase Decisions, Applied Economics , 30, 643-647. Johnson, F.R. and W.H. Desvousges (1997), “Estimating Stated Preferences with Rated-Pair Data: Environmental, Health, and Employment Effects of Energy Programs,” Journal of Environmental Economics and Management , 34, 79-99. Kahneman, D., P. Slovic, and A. Tversky (1982), Judgment under Uncertainty: Heuristics and Biases (New York : Cambridge University Press). Karni, E. and Z. Safra (1987), “Preference Reversal and the Observability of Preferences by Experimental Methods,” Econometrica , 55 675-685. King, G. (1989), Unifying Political Methodology: The Likelihood Theory of Statistical Inference , (New York: Cambridge University Press). Kriström, B. (1990), “A Non-Parametric Approach to the Estimation of Welfare Measures in Discrete Response Valuation Studies,” Land Economics , 66, 135-139. Kriström, B. (1997), “The Practical Problems of Contingent Valuation, in R.J. Kopp, W.W. Pommerhene, and N. Schwartz, eds., Determining the Value of Non-Marketed Goods (Boston: Kluwer). Lankford, R.H. (1985), “Preferences of Citizens for Public Expenditures on Elementary and Secondary Education,” Journal of Econometrics , 27, 1-20. Louviere, J.J. (1994), “Conjoint Analysis,” in Handbook of Marketing Research, R. Bagozzi, ed. (Oxford: Oxford University Press).46

Lohmann, S. (1994), “Information Aggregation Through Costly Political Action,” American Economic Review 84, 518- 530. Lunander, A. (1998), “Inducing Incentives to Understate and to Overstate Willingness to Pay within the Open-Ended and the Dichotomous-Choice Elicitation Format,” Journal of Environmental Economics and Management , 35, 88-102. Lupia, A. (1994), “Shortcuts Versus Encyclopedias—Information and Voting Behavior in California Insurance Reform Elections,” American Political Science Review , 88, 63-76. Machina, M.J. (1995), “Non-Expected Utility and the Robustness of the Classical Insurance Paradigm,” Geneva Papers on Risk and Insurance Theory , 20, 9-50. McConnell, K. E. (1990), “Models for Referendum Data: The Structure of Discrete Choice Models for Contingent Valuation,” Journal of Environmental Economics and Management , 18, 19-34. McDowell, I. and C. Newell (1996), Measuring Health: A Guide to Rating Scales and Questionnaires, 2nd ed. (New York: Oxford University Press). McFadden, D. (1974), “Conditional Logit Analysis of Qualitative Choice Behavior,” in Frontiers in Econometrics , P. Zarembka, ed., (New York: Academic Press). McFadden, D. (1994), “Contingent Valuation and Social Choice,” American Journal of Agricultural Economics 76, 689-708. McFadden, D. (1999) “Rationality for Economists?,” Journal of Risk and Uncertainty , 19, 73-105

. McLeod, D.M; Bergland, O. (1999), “Willingness-to-pay Estimates Using the Double-Bounded Dichotomous-Choice Contingent Valuation Format: A Test for Validity and Precision in a Bayesian Framework, Land Economics , 75, 115-125. Mitchell, R.C. and R.T. Carson (1986), "The Use of Contingent Valuation Data for Benefit -Cost Analysis in Water Pollution Control," report to the U.S. Environmental Protection Agency, September 1986. Mitchell, R.C. and R.T. Carson (1989), Using Surveys to Value Public Goods: The Contingent Valuation Method. Baltimore: Johns Hopkins University Press. Moulin, H. (1994), “Social Choice,” in Handbook of Game Theory with Economic Applications, R.J. Aumann and S. Hart, eds. (Amsterdam: North- Holland). Neil, H., R.G. Cummings, P.T. Ganderton, and G.W. Harrison (1994), “Hypothetical Surveys and Real Economic Commitments,” Land Economics 70, 145-154. Payne, S. (1951), The Art of Asking Questions (Princeton: Princeton University Press). Polasky, S., O. Gainutdinova, and J. Kerkvliet (1996), “Comparing CV Responses with Voting Behavior: Open Space Survey and Referendum in Corvallis Oregon,” paper presented at the annual U.S.D.A. W- 133 meeting, Jekyll Island, GA, February. Popkins, S. (1991), The Reasoning Voter (Chicago: University of Chicago Press).47

Posavac, S.S. (1998), “Strategic overbidding in contingent valuation: Stated economic value of public goods varies according to consumers expectations of funding source,” Journal of Economic Psychology ,19, 205-214. Randall, A. (1994), “A Difficulty with the Travel Cost Method,” Land Economics , 70, 88-96. Randall, A. (1996), “Calibration of CV Responses: Discussion,” in The Contingent Valuation of Environmental Resources: Methodological Issues and Research Needs, D.J. Bjornstad and J. R. Kahn (Brookfield, VT: Edward Elgar). Randall, A., B.C. Ives, and C. Eastman (1974), “Bidding Games for Valuation of Aesthetic Environmental Improvements,” Journal of Environmental Economics and Management 1, 132-149. Richer, J. (1995), "Willingness to Pay for Desert Protection," Contemporary Economic Policy , 13, 93-104. Romer, T. and H. Rosenthal (1978), “Political Resource Allocation, Controlled Agendas, and the Status Quo,” Public Choice , 33, 27-43. Rowe, R.D., W.D. Schulze, and W. Breffle (1996). “A Test for Payment Card Biases,” Journal of Environmental Economics and Management 31, 178-185. Samuelson, P.A. (1954), “The Pure Theory of Public Expenditure,” Review of Economics and Statistics 36, 387- 389. Satterthwaite, M. (1975), “Strategy-Proofness and Arrow Conditions: Existence and Correspondence Theorems for Voting Procedures and Welfare Functions,” Journal of Economic Theory 10, 187-217. Swait, J. and J.J. Louviere (1993), “The Role of the Scale Parameter in the Estimation and Use of Generalized Extreme Value Models,” Journal of Marketing Research , 30, 305-314. Scott, A. (1965), “The Valuation of Game Resources: Some Theoretical Aspects,” Canadian Fisheries Report 4, 27-47. Seip, K. and J. Strand (1992), “Willingness to Pay for Environmental Goods in Norway: A CV Study with Real Payment,” Environmental and Resource Economics, 2, 91-106. Smith, V.K. and W.H. Desvousges (1986), Measuring Water Quality Benefits (Boston: Kluwer). Sudman, S., N.M. Bradburn, and N. Schwarz (1996), Thinking About Answers: The Application of Cognitive Processes to Survey Methodology (San Francisco: Jossey-Bass Publishers). Sugden, R. (1999) “Alternatives to Neo-classical Theory of Choice,” in Valuing Environmental Preferences: Theory and Practice of the Contingent Valuation Method in the USA, EC, and Developing Countries , edited by I.J. Bateman and K.G. Willis. (Oxford: Oxford University Press). Tversky, A., P. Slovic, and D. Kahneman (1990), “The Causes of Preference Reversal,” American Economic Review , 80, 204-217. Varian, H. (1992), Microeconomic Analysis, 3rd. ed. (New York: Norton).