# PRIVATE GOOD VALIDITY EXPERIMENTS OF CVM 

**OPEN-ENDED AND CLOSE-ENDED WTP QUESTIONS:**

**COMPARING HYPOTHETICAL AND ACTUAL WILLINGNESS TO PAY**

John Loomis, Department of Agricultural and Resource Economics

Colorado State University, Fort Collins, CO 80523

Phone: (970) 491-2485; FAX(970) 491-2067

Thomas Brown and George Peterson, Rocky Mountain Forest and Range
Experiment Station

USDA Forest Service, 240 West Prospect Street, Fort Collins, CO 80523

I. TESTING CRITERION VALIDITY OF CONTINGENT VALUATION

Criterion validity tests of CVM compare hypothetical WTP to actual cash
payments. Past criterion validity tests of CVM have been of two basic
types: field experiments and laboratory experiments. Lab experiments
comparing actual cash and hypothetical WTP have the advantage of careful
control of procedures, use of a well defined, deliverable good. Most lab
experiments have used common market goods, such as a chocolate bar
(Kealy, et al. 1988), a houseplant (Boyce, et al., 1991), a painting or
map (Neill et al. 1994), a juicer, calculator and box of chocolates
(Cummings, et al., 1995) and an information booklet (Cummings, et al.,
199?).

Lab experiments have relied upon common market goods because they are
deliverable and familiar, based on the belief that if CVM cannot get it
right for these goods, there is much less hope for more complex and
unfamiliar environmental services (Cummings, et al. 1995). However,
designing experiments that use market goods may be plagued by several
problems. First, is the possibility that in stating their WTP in the
hypothetical market, some individuals are stating what they believe a
fair price is or what they guess the market price to be, rather than
what they would pay. This reduces the insight these experiments provide
to estimating the value of public goods for which no obvious price
exists. Second, some subjects may answer what they would pay if they
were in the market for such a good, when in fact they are not at the
present time. Further, respondents may project into a less constrained
environment when answering the hypothetical WTP question as compared to
the actual payment question. For example, they might be stating what
they would pay after payday. These conditions often do not parallel the
cash experimental treatments where respondents are asked to pay at that
very moment or required to take out a loan that must be repaid within a
few days (Neill, et al. 1994:149).

The goals of our research are to: (1) compare estimates of hypothetical
and actual WTP for a market good using both open-ended and dichotomous
choice WTP question formats; (2) test whether some of the disparity
between hypothetical and actual WTP can be reduced by explicitly
instructing respondents in the hypothetical treatment that we do not
wish their estimate of a fair market price;

II\. RESEARCH DESIGN

A laboratory experiment was designed to compare estimates of WTP
elicited using open-ended and dichotomous choice question formats in
both hypothetical and real markets for the same good. This experiment
has five treatments:

\(a\) WTP-OE(hyp: no reminder): hypothetical WTP asked as in a standard
CVM survey, using wording (given below) similar to Neill, et al.

\(b\) WTP-OE(hyp:reminder): hypothetical WTP asked after subjects are
reminded not to give what they think a fair price is or what it sells
for and to act as if they were in a real market with their real budget.

\(c\) WTP-OE(actual): actual WTP in the form of cash, check, or
promissory note.

\(d\) WTP-DC(hyp:reminder): hypothetical WTP asked after subjects are
reminded not to decide whether the bid is a fair price or what it sells
for and act as if they were in a real market with their real budget

\(e\) WTP-DC(actual): response in actual, binding market, where they
would have to actually pay their bid amount with cash, check or
promissory note.

The first three hypotheses tests involve criterion validity:

\(1\) Ho: WTP-OE(hyp:no reminder) = WTP-OE(actual)

\(2\) Ho: WTP-OE(hyp:reminder) = WTP-OE(actual)

\(3\) Ho: WTP-DC(hyp:reminder) = WTP-DC(actual)

One objective here is to test whether efforts to educate respondents in
the hypothetical market that we desire their immediate value as if they
were in a real market and not some measure of fair price reduces the
disparity between hypothetical and actual cash values as compared to
Neill, et al.\'s experiment. After we designed and implemented this
experiment, we found our approach is similar to the ??Cummings and
Osborne?? \"cheap talk\" experiments.

III\. EXPERIMENTAL DESIGN

[Subjects:]{.underline} University clerical and administrative staff in
academic and non-academic units were recruited and paid \$20 for
attending a 45 minute session on campus. The same researcher conducted
all of the sessions, closely following a script for each session that
was identical except for treatment effects.

[Nature of the Good:]{.underline} The good chosen for the experiment-an
art print-reflected several desirable features. First, art prints are
infrequently purchased and different prints sell at quite different
prices, so people would not be likely to know the market price of a
given print. The objective was to minimize the likelihood that the
respondent would simply try to state the market price. Second, the good
had readily observable characteristics, so there is minimal ambiguity in
terms of what the product is. Finally, we wanted a good that is not too
expensive so it could be paid for in cash or from the current balance in
their checking account.

[Setting of Experiments:]{.underline} All the sessions were held in a
classroom. Individuals were instructed to read and complete the bid
submittal page. When everyone had finished, the sheets were passed
forward. In the actual treatments the winner was announced and asked to
come forward to complete his or her purchase in front of the group, but
the winning price was not announced. Individuals were allowed to pay
with cash or check or sign a promissory note payable within three weeks.

[Wording of WTP Questions]{.underline}

WTP-OE(hyp:no reminder): In session #1, the wording of the WTP(h: no
reminder) question was \"You are being asked to participate in a
[hypothetical]{.underline} sealed bid auction for this print. We would
like to know the maximum amount of money you would pay to take this art
print with you at the end of this session, [if]{.underline} this one art
print were actually for sale, and you would have to pay by August 19,
1994.

Now please write down the maximum dollar amount you would be prepared to
pay for this art print.

I would bid \$\_\_\_\_\_\_\_\_\_\_. This wording was patterned after
Neill, et al.

WTP-OE(hyp:reminder): In session #2, the wording of the WTP(h:with
reminder) question was \"You are being asked to participate in a
[hypothetical]{.underline} sealed bid auction for this print. We would
like to know the maximum amount of money you would pay to take this art
print with you at the end of this session, [if]{.underline} this one art
print were actually for sale. At this time in the survey, we are NOT
asking what you think the art print might sell for in a store or what
you think its fair price is. Rather, we want to know the maximum amount
of money that you would honestly be prepared to pay right now to buy the
art print you are being shown if you would really be required to pay
your bid amount with cash, write a check today, or sign a Promissory
Note payable on or before August 19. Please take into consideration your
budget and what you can afford to pay. If what you would pay is
different from what you judge a fair price to be, that is OK. We want to
know what you would actually be prepared to pay for the art print. Take
a few moments to think about what you honestly would be prepared to pay
for this art print if it were being offered for sale to you today and it
would go to the highest bidder. Although the question is hypothetical,
we want you to answer as if it were for real - as if you were
participating in a real sealed-bid auction and would really have to pay
your dollar amount if you were the highest bidder.

Now, please write down the maximum dollar amount you would be prepared
to pay for this art print.

I would bid, and would really be prepared to pay,
\$\_\_\_\_\_\_\_\_\_\_.

As can be seen, the reminder version attempted to more fully place the
individual in the frame of mind of the real market situation, without
actually requiring them to pay. This statement was developed after
discussions with pre-test participants indicated that they were in a
different frame of mind when answering the hypothetical WTP questions as
compared to a follow-up actual cash question. Second, we wanted
individuals to report their WTP for the print rather than attempt to
estimate what they thought a reasonable price would be in a store.

WTP-OE(actual): Wording of the actual cash WTP question was: As part of
this experiment, we are now going to conduct a [real
auction]{.underline}. This art print will be sold to the highest bidder
here today. Only one of these prints will be sold at this auction. After
all bids have been collected, the person who is the highest bidder will
be announced and he or she will be obligated to purchase the print at
his or her bid price. We will accept cash or check for your purchase. We
understand that you may not have anticipated the need to bring cash or
your checkbook with you today, so we will also accept a signed
Promissory Note payable on or before August 19. In any case, the highest
bidder will be required to pay his or her bid amount and will then be
able to take the art print home with him or her at the end of this
session. Please understand you are participating in a [real
auction.]{.underline} Now take a few moments to determine the maximum
dollar amount that you are prepared to pay for this art print. What is
the most you are prepared to pay for this art print? I bid
\$\_\_\_\_\_\_\_.

WTP-DC(hyp:reminder): The wording of the dichotomous choice question in
the hypothetical market treatment was similar to the WTP-OE
(hyp:reminder) except stating they would pay their preassigned dollar
amount. The reminder statement was identical to the WTP-OE(hyp:
reminder).

WTP-DC(actual): The wording of the real market dichotomous choice
question was identical to the WTP-OE(actual) except stating they had to
pay their preassigned dollar amount.

IV\. RESULTS OF HYPOTHESIS TESTS

Table 1 reports mean and median WTP\'s for the five sessions. The
independent means for the open-ended WTP responses are \$42, \$26 and
\$14 for WTP-OE(hyp:no reminder), WTP-OE(hyp:reminder) and
WTP-OE(actual), respectively. The WTP-DC(hyp;reminder) mean is \$28 from
the logit model and \$33 when estimated non-parametrically (Kristrom,
1990). The WTP-DC(actual) is \$11 using both estimation techniques.

[Results of Formal Hypothesis Tests]{.underline}

The Mann-Whitney-Wilcoxon two-sample rank test is used to test whether
there is a significant difference in the medians. This test was used in
lieu of the Student\'s t-test due to non-normality of the data. As shown
in the first three rows of Table 2, both tests reject equality of
hypothetical and actual WTP using open-ended or dichotomous choice. As
shown in Table 1, the primary effect of including the reminder was to
reduce the size of the disparity.

V. CONCLUSION

Our results are consistent with those of Neill, et al. (1994) and Kealy,
et al. (1988) in that statements of hypothetical WTP are typically
statistically greater than actual cash WTP. Excluding WTP\'s over
\$1,000, the Neill et al., map experiment produced differences between
hypothetical and actual WTP that were about 9:1. Our most comparable
treatment (no reminder), which used nearly the exact wording of Neill,
et al., produced differences in WTP that were about 3:1. In our
hypothetical session with the reminder, the difference between the
hypothetical and real WTP is 1.8:1. Thus reminding respondents: (1) not
to bid what they thought the good might sell for in a store; (2) to act
as if they were in a real market; and (3) to take their household budget
and available funds into consideration when bidding reduced the
disparity of hypothetical and actual WTP. This reminder also resuledt in
a significant improvement in the correlation of actual to hypothetical
WTP in paired samples. Further, a regression analyses showed that the
reminder lessened the association of hypothetical WTP with perceived
market price. Without the reminder, the highest bids were \$150 and
\$400, and inclusion of these two high bids explains much of the
difference between our two hypothetical treatments. An advantage of the
reminder is that it apparently reduces the tendency of some respondents
to give unrealistically large WTP bids.

Since we found that perceived market price of the print was
significantly related to hypothetical WTP without the reminder some
subjects apparently attempted to rely on their estimate of what they
thought the price of the good was to estimate their WTP in the
hypothetical treatment. This finding raises the possibility that
experiments using market goods may not provide an unambiguous test of
criterion validity of CVM for non-market goods, as has been previously
assumed.

Using the dichtomous choice question format with the reminder,
hypothetical WTP was statsitically larger than actual WTP, with a ratio
of 2.5, larger than the open-ended ratio. However, both the open-ended
and dichotomous choice question format provided estimates of
hypothetical WTP that were not statistically different from each other
and and this was also true of actual cash WTP estimates from the two
question formats.

What can be learned about validity experiments, in particular, and CVM
survey design, in general, from experiments such as ours and Neill, et
al.\'s? One important lesson is to continue to debrief and probe
respondents about the differences in their decision processes in
hypothetical versus actual cash decisions. Using this knowledge,
statements to combat the hypothetical nature and place them in a real
market frame of mind can be developed to better improve the match
between intended behavior and actual behavior. Improving the linkage
between what people say and what they would do will go along way toward
increasing the acceptability of CVM derived results in natural resource
decision making. The learning design of Bjornstad, Cummings and Osborne
provides significant promise in this regard.

REFERENCES

Bjornstad, D., R.Cummings and L. Osborne A Learning Design for Reducing
Hypothetical Bias in the Contingent Valuation Method. Oakridge National
Lab, Oakridge, TN.

Boyce, Rebecca, Thomas Brown, Gary McClelland, George Peterson and
William Schulze. 1992. An Experimental Examination of Intrinsic Values
as a Source of WTA-WTP Disparity. American Economic Review
82(5):1366-1373.

Cummings, Ronald, Glenn Harrison and E.E. Rutstrom. 1995. Homegrown
Values and Hypothetical Surveys: Is the Dichotomous Choice Approach
Incentive Compatible? American Economic Review 85(March):260-266.

Cummings, R., S. Elliot, G. Harrison, and J. Murphy. 199?. Are
Hypothetical Referenda Incentive Compatible. Journal of Political
Economy

Gibbons, Jean Dickinson. 1993. Nonparametric Statistics, An
Introduction. California: Sage Publications.

Kealy, Mary Jo, John Dovidio and Mark Rockel. 1988. Accuracy in
Valuation is a Matter of Degree. Land Economics 64(May): 158-171.

Kristrom, B. 1990. A Non-Parametric Approach to the Estimation of
Welfare Measures in Discrete Choice Response Valuation Studies. Land
Economics 66(2):135-139.

Neill, Helen, Ronald Cummings, Philip Ganderton, Glenn Harrison and
Thomas McGuckin. 1994. Hypothetical Surveys and Real Economic
Commitments. Land Economics 70(May):145-154.

TABLE 1 Comparison of Hypothetical and Actual WTP

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Session Sample Mean (Median) WTP by Treatment

Size \[95%CI\'s\]

WTP-OE WTP-OE WTP-OE WTP-DC WTP-DC Ratio

Hypothetical Hypothetical Actual Hypothetical Actual WTP(h)/

without w/reminder w/reminder WTP(a)

reminder

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

#1 35 \$42 (25) 3.0

\[20-65\]

#2 33 \$26 (20) 1.8

\[19-33\]

#3 32 \$14 (10)

\[12-16\]

#4 52 \$28(28) 2.5

\[20-37\]

#5 55 \$11(9)

\[6-22\]

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

The reminder treatment involved reminding respondents not to report a
fair price for the good and to act like they would if they actually had
to pay for the art print.

Table 2 Results of Hypothesis Tests

+-------------------------------------+----------------+--------------+---+
| Hypothesis                          | Mann-Whitney   | Mean WTP     |   |
|                                     | Test of        | CI\'s        |   |
|                                     | Medians        | Overlap?     |   |
+-------------------------------------+----------------+--------------+---+
| WTP-OE(Hyp-no reminder)=            | P=.0017        | No           |   |
|                                     |                |              |   |
| WTP-OE(actual)                      |                |              |   |
+-------------------------------------+----------------+--------------+---+
| WTP-OE(Hyp-reminder)=               | P=.09          | No           |   |
|                                     |                |              |   |
| WTP-OE(actual)                      |                |              |   |
+-------------------------------------+----------------+--------------+---+
| WTP-DC(hyp:reminder)=               | P=.006         | No           |   |
|                                     |                |              |   |
| WTP-DC(actual)                      |                |              |   |
+-------------------------------------+----------------+--------------+---+
| WTP-DC(actual)=                     | P=.77          | Yes          |   |
|                                     |                |              |   |
| WTP-OE(actual)                      |                |              |   |
+-------------------------------------+----------------+--------------+---+
| WTP-DC(hyp:reminder)=               | P=.48          | Yes          |   |
|                                     |                |              |   |
| WTP-OE(hyp:reminder)                |                |              |   |
+-------------------------------------+----------------+--------------+---+