### MARKET MODELS, PROTEST BIDS, AND OUTLIERS IN
CONTINGENT VALUATION 

### ByGregLindsey

~

ABS'mACT: Responsesto willingness-to-pay questionsincontingentvaluation(CV) surveysarebeingused withincreasingfrequencyto obtain estimatesofthe value ofpublicgoods, suchasimprovedwater quality, foruse inbenefit-costanalyses. Analystsmustdecidewhethertheappropriate conceptualmodelforanyparticular applicationisa private or a politicalmarket. When estimatingwillingness-to-pay andtotalbenefits,analystsmustdecidewhethertocensoroutliersandcertainzero responses,calledprotest bids,fromdata sets. Issuesassociatedwithidentification ofprotest bidsare reviewed, and alternate methods for the treatment ofoutliers
arediscussed.The significance ofthesemethodologicaldecisionsisillustratedwith
anexampleinvolving

willingness

topayfornewstormwaterprogramsinBaltimore County, Md. to help achieveChesapeake Baynutrient-reduction objectives.Itis arguedthat decisionsconcerninganalysisoftroublesomeresponsesshouldbeconsistentwiththe choiceofmarket modelforthe application.

INTRODUCTION

Since the U.S. Water Resources Council 

### (Economic 1983), the Army Corps of Engineers (Moser and Dunning 1986), and other agencies have sanctioned the use of contingent valuation (CV) methodology in benefitcostapplications,surveyshave been used withincreasingfrequencyto obtain estimates of benefits for public goods, such as water quality. Recent, wellknown examples include Carson and Mitchell (1986) and Smith and Desvousges (1986). Some researchers, however, are skeptical about the usefulnessofthe information collectedin surveys, simplybecause itisso difficult to describe all the relevant parameters of the item to be valued (Fischhoff and Furby 1988). Administrators have long recognized, for example, that willingnessto pay (WTP) for flood control is greater after a significant flood event (Stitt 1986). It is questionable whether preflood surveys can capture postflood WTP. Similarly, pre- and postdrought surveys about WTP for a reliablewater supply might obtain very different results. Hobbs et al. (1989) concludes that CV is at the initial implementation state, and that many methodologic problems must be solved before these tools can be routinely used but that even attempts to place values on human lives can be useful. Thispaper addresses one ofthe methodologicconsiderationsthat confront analysts who use CV, namely, how decisions to treat outliers and dertain

zero bids can affect estimates of total benefits. Analysts know that some respondents will be unwilling or unable to participate meaningfully in surveysand willprovideunreasonable or arbitraryresponses to questions about WTP (Smith and Desvousges 1986). Hence, they frequently censor so-called "protest zeroes" and outliers. Protest zeroes are statements of zero WTP from individuals who presumably value the good positively. Outliers are any bids that seem unlikely given presumptions about the underlying 

### distAsst. Prof., Ping. Program, School of Public and Envir. Affairs, Indiana Univ., BS 4068, 801 West Michigan, Indianapolis, IN 46202. Note. Discussion open until July 1, 1994.To extend the closingdate one month,
awrittenrequest mustbe filedwiththe ASCEManagerofJournals. The manuscript for this technical note was submitted for review and possiblepublication on April 23, 1992. This technicalnote is part ofthe

### Journal ofWaterResourcesPlanning and 

### Management, Vol. 120, No. 1, January/February, 1994. 9 ISSN 0733-950X/ 94/0001-0121/$1.00 + $.15 per page. Technical note No. 3926.

121

### tribution ofbids. Techniques used to treat these bids are important because

they can affect estimates of aggregate WTP (Milon 1989). It is argued here that the analyst's choice of techniques should follow from the conceptual model for the CV application.

MODELS FOR CONTINGENT VALUATION APPLICATIONS


### The conceptual model that has guided the development of CV has been the private market, and CV exercises typically have been designed to emulate markets for private goods and services. The premise of CV is that surveys can be used to elicit accurately maximum WTP for public goods; it is assumed that bids represent what the good is worth to the respondents. Individual bids must be aggregated; the sample mean bid is usually extrapolated to the total population to obtain estimates oftotal WTP. A decision rule, such as positive net present value in which total benefits are based on
an estimate of mean WTP, implies that projects valued highly by a small minority of the population could be approved. If, however, certain bids do not represent true valuations, the model breaks down, and the troublesome bids must be censored. This has led to the development of techniques for differentiatingprotest zeroes fromvalid zeroes and to debate overthe treatment of outliers. An alternate conceptual model for CV experiments is a political market (Mitchell and Carson 1989). This is generally more conservative: the appropriate decision rule is the median bid, which, given the shape of most WTP distributions, means that estimated WTP will be lower. The numbers of zero bids and outliers still are important because they can affect the estimate of the median. Conceptually, however, the analyst is faced with somewhatdifferent problems. Rather than being concerned mainlywhether

a zero response represents an individual's true valuation of a public good, the analyst must determine whether a zero bid accurately reflects how a respondent might react if confronted in a referendum with a tax proposal. Individuals may be unwilling to pay more for an environmental program--
even though they value that program highly--because, as is the case with most public goods, they have no direct control over the establishment of prioritiesorthe allocation offunds to individual items in the bundle ofgoods paid for with tax dollars. Hence, in a political market, more zeroes will be accepted at their face value. With outliers, because the exact value of high bids atthe tailofthe distribution isirrelevanttotheestimation ofthe median bid, the method of treatment becomes less important. The political and private market models can be thought of as end points

on a continuum; estimates of WTP consistent with each establish bounds

on the range of benefits. From a methodological perspective, the implications of choosing a political market as a model are intuitively clear and desirable: the problems of controlling for protest bids and outliers are reduced. The importance of consistency between the analysts' conceptual models and choice of techniques is illustrated in the following application.

APPLICATION: WTP FOR STORM-WATER MANAGEMENT


### A CV survey was administered by mail in August 1989 to 1,600 homeowners in Baltimore County, Md. to determine their WTP for storm-water programs to help achieve Chesapeake Bay nutrient-reduction Objectives. The recipients were selected randomly from 133,344 owner-occupants listed in county property-tax files. Fifty-eight percent of the questionnaires were

122returned. The response rates to individualquestions on the completed questionnaires ranged from 77% to 97%. The sample is generally believed to be representative ofhomeowners in Baltimore County. Details ofthe survey and additional information about results are provided in Lindsey (1992). The elicitation question was phrased: What is the most you would be willing to pay each year in storm-water utility charges before you would vote no in the referendum? The answer option consisted of a matrix of values ranging from $0.00 to $1,000.00 called a payment card. Approximately 89% of the respondents answered the WTP question. The distribution of WTP (n = 824) responses is nonnormal (Lilliefors statistic = 0.298), positively skewed (skewness coefficient = 5.742), and leptokurtic (kurtosis coefficient = 52.607). The unadjusted, mean annual bid is $42.00 and the median is $20.00. The mode is $0.00 (33% of the responses). The highest bid was $1,000.00; 26 bids were above $200.00.

TREATMENT OF PROTEST ZEROES AND OUTLIERS

To identify protest zeroes, researchers ask respondents to explain their

zero bids. Choices of answers typically include: "that is what it is worth to me"; "did not want to place dollar value"; or "am opposed to all new taxes." Zeroes explained by reasons other than "that is what it is worth to me" or "all I can afford" are often classified as protest bids. In this application, homeowners were provided eight alternative explanations for zero bids. A plausible approach to classification of each under both private and political market models is presented in Table 1. For some, the appropriate classification is not obvious. Explanations 02, 03, and 06 (did not want to place dollar value, objected to way question was presented, and not enough information) each imply thatrespondents might have answered differentlyhad they more information

or if the question had been structured differently. Hence, an analyst could

TABLE 1. Survey Question: If You Are Not Willing to Pay any New Charges to Control Storm Water and Improve Water Quality in Chesapeake Bay, What Is the One Most important Reason You Answered the Way You Did?

lumber (1) 01
02
03
04
05
06 07 08 Number of

Choice of answers answers Percent (2) (3) (4) Thatiswhatitisworth 9 3 No tome Didnotwanttoplace 15 6 Yes dollarvalue Objectedtowayquestion 3 1 Yes
waspresented Toomuchwastein 75 28 Perhaps government Amopposedto allnew 51 19 Perhaps taxes Notenoughinformation 13 5 Yes Industryshouldpay 66 25 Perhaps Other(pleaseexplain) 36 13 Perhaps Totalresponses 268 a 100 -- Classified as Protest Bid?
In model of private-goods In model of market political market (5) (6)

No
Perhaps

Perhaps

No

No Perhaps No No

aThere were 274 bids of zero WTP; 98% of the respondents who stated zero willingness to pay answered this question.

123conclude that zeroes explained by these answers are not accurate representations of true WTP and that, regardless of the conceptual model, these shouldbe classified asprotest zeroes. Zero, however, maybe the most likely bid from an individual who is uninformed or uncomfortable with a question

on a referendum ballot. Hence, the rationale for second-guessing the respondent is weaker if the conceptual model is a political market. Twenty-fivepercent ofthe respondents bid zero because "industry should pay" (answer 07). Interpretation of this explanation is more troublesome for it provides no insight into whether respondents value the good, only that they believe someone else should pay. Classification of these bids as protest zeroes seems plausible if the conceptual model is a private market and the objective of the analysis is to determine true WTP. It seems inappropriate, however, if the model is a political market and one believes that individuals who say that someone else should pay for a program would vote zero because of their convictions. Almost half (47%) explained their zero bids by responding that there is "too much waste in government" or that they are "opposed to all new taxes" (answers 04 and 05, respectively). These responses also provide no information about true valuation of storm-water programs, but they do provide information about preferences for more government programs. Treatment of these as protest zeroes may be acceptable if the conceptual model is a private market. If the model is a referendum, however, there is little justification for censoring them. The least ambiguous response is "that is what it is worth to me" (answer 01). Only nine individuals explained their bids with this answer. This is somewhat troubling: a literal interpretation is that just 3% of the zero bids

are accurate. Given the eight possible explanations, there are 256 (28) possible subsets of protest zeroes and potential estimates of WTP. The effects of different approaches to classification are illustrated with four subsets in Table 2. The benchmark from which these approaches are evaluated involves extrapolation ofthe sample mean ($42.00) and median ($20.00) without adjustment for protest zeroes. Estimates of total annual WTP corresponding to these point estimates are $5,600,000 and $2,700,000, respectively. Another option would be to classify asprotest bids onlythose respondents who stated explicitly that they did not want to value the good, that they objected to the question, or that they lacked sufficient information (answers 02, 03, 06, respectively). Deletion of these 31 bids increases the estimates

TABLE 2. Effects of Deleting Protest Bids on WTP

Distributionstatistics (t) Numberofcases Median($) Mean($) Variance Standarderror TotalWTP:extrapolatingmedian TotalWTP:extrapolatingmean POLITICAL MARKET PRIVATE MARKET Answers Deleted as Protest Bids None (benchmark) (2) 824 20 42.00 5,724 2.6 $2,700,000 $5,600,000 02-03.06 02-03.06-07

(3) (4) 793 727 25 43.60 47.60 5,878 6,223 2.7 2.9 $3,300,000 $5,800,000 $6,300,00 02-08 (5) 665
61.20 a 7172 3.0 $8,200,000 aSignificantly

differentfrombenchmarkandothercasesat95%confidence interval.

124

### of mean and median WTP to $43.60 and $25.00, respectively (Table 2).

Correspondingestimates oftotal WTP increase to $5,800,000 and $3,300,000. A third approach would be to delete, in addition to these, the 66 zero bidsexplainedbythe response "industryshouldpay." This approach (which requires the analyst to exercise greaterjudgment and, as noted previously,

seems appropriate onlyfor a private market model), increases estimates of annual mean and total WTP to $47.60 and $6,300,000, respectively (Table 2). An approach that implies strict adherence to a private goods model involvesdeletion ofall the zero bids, exceptthose explained by "that is what it is worth to me." This increases estimates of the annual mean and total WTP to $61.20 and $8,200,000, respectively (Table 2). The estimate is significantlydifferent from the estimates in the other cases, the differences among which are not significant. What is clear from this example is that analystswhoseframe ofreferenceis a politicalmarket are requiredto make fewer interpretations of data. Outliers pose comparable problems. Analysts must first define and then decide what to do with them. Researchers have identified outliers by adhoc, arbitrarystandards [e.g., asbids greaterthan 10standard deviations from the sample mean bid (Rowe et al. 1980) or as bids greater than some percentage of a respondent's annual household income]; through the use ofa robust statistical estimator, such as the a-trimmed mean (Mitchell and Carson 1989); or through the use of regression diagnostics, such as the DFBETA statistic (Smith and Desvousges 1986), or the analysis of studentized residuals. Use of a-trimmed mean procedures (Appendix I) with CV data poses severalproblems, mainlybecause bids are constrainedto be zero or greater. It is not uncommon that, as in this case, a bid of zero is the mode. To use the procedure, then, requires judgments about the zeroes to be deleted. Anotherprobleminvolvessettingthe a-level. Analyststypicallyrecommend values of5-10%, although this is subjective. Other problems occur due to clusters of bids at round numbers. The DFBETA statistic is a measure of the influence of an observation

on the coefficients in an ordinary least-squares (OLS) regression (Belsley et al. 1980). The approach, which involves calculating the change in each estimated coefficientthat results from deletion of an observation, can become cumbersome if a regression equation includes more than one independent variable (Moser and Dunning 1986). In addition, the overall effect of eliminating observations is difficult to determine because the elements ofthe statistic are not commensurable (Fox 1984). Another regression procedure for determining outliers involves analysis of the studentized residuals from a bid function (Belsley et al. 1980; Fox 1984;Wilkinson 1988). Studentizedresiduals are estimatedbystandardizing the residuals from OLS regression and "successively omitting each observation, calculating its residual based on the regression coefficientsobtained forthe remaining sample, and dividing the resulting residual byits standard error" (Fox 1984) (see Appendix II). With this approach, model specification and methods of compensating for item nonresponse are important procedural considerations. Once outliers have been identified, the problem ofwhat to do with them remains. Most analysts censorthem. It is not difficult,however, to envision
cases in which the choice of technique could become a factor in whether a hypothesis is accepted or rejected or whether a project isjudged efficient.

125In controversial applications, if outliers are deleted simply because the residuals from a regression are unusually large, critics can charge that the analyst has censored from the sample those individuals who do not conform to narrow conceptions of economic behavior. As with protest bids, one perspective is that treatment of outliers is more important when the conceptual model forthe application is a private market (i.e., when thepoint estimate ofinterest isthe mean rather than the median). Adoption of a political market minimizes the problem of outliers if, as will be the case in most applications, the outliers comprise high bids at the end of the distribution and it is assumed that the bids accurately reflect some positive WTP greater than the median value. This is because the absolute magnitudes of bids greater than the median do not influence the estimate of the median value. The effects of three different approaches to controlling outliers in Baltimore County stormwater survey are illustrated in Table 3. The ad hoc procedure of culling bids greater than 10 standard deviations above the

mean (or of bids greater than 0.5% of reported income) results in deletion of just one $1,000 bid. The resultant mean ($40.80) is not significantly different than the benchmark mean ($42.00, given in Table 3). Selection of the cutoff is arbitrary, however. Use of the a-trimmed mean approach (a = 4%) results in deletion of 33 bids greater than $150.00 (and 33 zero bids). The a-level of 4% was determined by inspection; zero bids were deleted randomly. The resultant estimate of mean willingness to pay ($32.30) is significantly different than the

mean estimated from the uncensored data (Table 3). The third approach involves identification of outliers based on the distribution of studentized residuals from an OLS regression. Analysis of studentized residuals for the bid function presented in Table 4 results in the identification of26outliers, each ofwhich isgreater than orequal to $200.00. Deletion of these bids yields a mean annual WTP of $32.50, which is also significantly different than the benchmark (Table 3). As noted, however, the decision to censor responses that do not fit a hypothesized model can be problematic. In this case, it is conceivable that some individuals would be willing to pay $200,00 or more annually to clean up urban storm water. Under a private-market model, a reasonable approach with these data might be to retain all zero bids but censor outliers identified in a regression framework. This results in an estimate of annual mean WTP of $32.50,

TABLE 3. Effects of Deleting Outliers on WTP

Distribution statistics (1) PRIVATE MARKET Technique Used to Identify Outliers

Political market (2) ,Bench mark

(3) Outliers =

bids > 10 SD
> Mean WTP (4) cx-trimmed

mean (4%) (5) Studentized

residuals (6) Numberofcases 824 824 823 759 798 Median/mean 20 42.00 40.80 32.30 ~ 32.50

a Variance -- 5,724 4,613 1,502 1,709 Standarderror -- 2.6 2.4 1.4 1.5 TotalWTP $2,700,000 $5,600,000 $5,400,000 $4,300,000 $4,300,000 aSignificantlydifferentfrombenchmarkat95%confidenceinterval.

126TABLE 4. Ordinary Least-Squares Bid Function for Identifying Studentized Residuals

Variablea Coefficient T-Statistic



### (1) (2) (3)

Constant Household income Years ofeducation Use ofthe bay foroutdoor recreation (dummy) Ranking ofbaywater quality Ageover 65 (dummy) Rating oflikelihood ofsuccessofbayprograms Goal statement (dummy) User charge payment vehicle (dummy) Paymentreminder (dummy) Questionnaire return date -45.1498 0.0005 2.7081 7.5159 -0.1932 -4.0902 4.8946 1.3508

- 1.8356

- 8.5265 0.0227 -3.11

5.60 3.22

1.62 -0.17

- 0.69 4.86 0.30 -0.41 -1.91 0.10 "Dependent variable is the willingnessto payfor storm-water programs. Note: adjusted R2 = 0.104; F statistic = 11.75;and N = 924.

which corresponds to an estimate of annual benefits for all owner-occupants of $4,300,000. The rationale for retaining all zeroes is that, except for the extreme case in which all zeroes are censored, except those explained by

answer 01 ("that is what it is worth to me"), estimates ofWTP do not differ significantly. This judgment is pragmatic and involves balancing the value of the "better" estimate that might be obtained if some subset of zeroes (e.g., those associated with answers 02, 03, and 06) were censored against the loss of information that occurs with the decision to censor. A rationale for using regression to identify outliers is that the approach is consistent with a private-market model. If, alternatively, a political market is assumed, the issues posed byprotest

zeroes and outliers are reduced. Ifno outliers are censored, the median bid

is $20.00 or $25.00 per year, depending on assumptions made about zero bids. Corresponding estimates oftotal WTP are $2,700,000 and $3,300,000, respectively.

SOME OBSERVATIONS

It is clear that the analyst's frame of reference or conceptual model for analysismaydictate use ofprocedures that, when implemented, significantly affect estimates ofWTP. Adherence to a private-market model rather than

a political market model will generally result in a higher estimate ofWTP. Adoption of a political market will require the analyst to make fewer interpretations ofambiguous responses. Although there always willbe ~imbiguity in how to classify certain responses, the issue in some applications may be moot if alternate classifications do not significantly change estimates of WTP. Careful presentation of both assumptions and methodology is essential to inform decision-makers adequately. Planners should take care that analytic techniques are consistent with conceptual models for any particular CV application.


### 127APPENDIX I. a-TRIMMED MEAN

The a-trimmed mean is defined by

= S(nct)+l -~ . . . 71- 

## X[i,t_(not)]

n - 2(mx)

................................ (1)

where Xi = set of ordered WTP observations; i = 1

. . . n; n = number of observations in the sample; and ct = percentage; 0.0 -< et < 0.50.

APPENDIX II. STUDENTIZED RESIDUALS

Studentized residuals follow a t-distribution with N-k-2 degrees of freedom and can be estimated by

~/n - k- 2 E7 = L

.................................. (2)

~/n - ~ - 1 - 

## E~

where/~i = least-squares residual standarized bydividingeach least-squares residual by its estimated standard deviation; E* = studentized residual; n

= number of observations; and k = number of regressors excluding the constant.

APPENDIX III. REFERENCES Belsley,D. A.,Kuh,E., andWelsch,R. E. (1980). Regressiondiagnostics: identifying influential data and sources ofcolinearity. John Wiley and Sons, Inc., New York, N.Y. Carson, R. T., and Mitchell, R. C. (1986). "The value of clean water: the public's willingnesstopayforboatable,fishable,andswimmablequalitywater."Discussion paper QE85-08, Resources for the Future, Washington, D.C. Economic and environmental principles and guidelinesfor water and related land

resources implementation studies. (1983). U.S. Water Resour. Council, Washington, D.C. Fischhoff, B., and Furby, L. (1988). "Measuring values: a conceptual framework for interpretingtransactions with specialreference to contingent valuation ofvisibility."J. Risk and Uncertainty, 1(2), 147-184. Fox,J. (1984). Linear statistical models and related methods. John Wileyand Sons, Inc., New York, N.Y. Hobbs, 13. F., Stakhiv, E. Z., and Grayman, W. M. (1989). "Impact evaluation procedures: theory, practice, and needs." J. of WaterResour. Ping. and Mgmt., 115(1), 2-21. Lindsey, G. (1992). "An experiment in contingent valuation: willingnessto pay for stormwatermanagement," PhD thesis,TheJohns HopkinsUniv., Baltimore,Md. Milon, J. W. (1989). "Contingent valuation experiments for strategicbehavior."J. Envir. Economics and Mgmt., 17(3), 293-308. Mitchell,R. C., and Carson, R. T. (1989). Using surveys to valuepublic goods: the contingent valuation method. Resources for the Future, Washington, D.C. Moser, D. A., and Dunning, M. C. (1986). "Volume II: a guide for using the contingent value methodology in recreation studies." National economic development procedures manual--recreation, U.S. Army Corps of Engrs., Inst. for Water Resour., Fort Belvoir, Va. Rowe, R. D., d'Arge, R. C., and Brookshire, D. S. (1980). "An experiment on the economicvalue ofvisibility."Envir. Economics and Mgmt., 7(1), 1-19.

128

### Smith, V. K., and Desvousges, W. H. (1986). Measuring water quality benefits.



### Kluwer-NijhoffPublishing, Boston, Ma. Stitt, T. A. (1986). "Establishing a storm water managementutility." Public Works, 

### 117(9), 126-127, 150, 152, 154, 156. Wilkinson, L. (1988). SYSTAT: thesystemfor statistics. 

### SYSTAT, Inc., Evanston, IU.

129