**Laboratory and Field Research: Complements and/or Substitutes**[^1]

**By**

**David S. Brookshire**

**Steven Stewart**[^2]

**October 1, 1998**

Over the years a literature has developed that addresses issues relating
to external validity or parallelism of laboratory and field
research.[^3] This effort began primarily with respect to the use of
student samples in laboratory market settings and whether the results
that are obtained in the laboratory would have been observed with
non-student samples. With regard to non-market elicitation, the notion
of parallelism relates to two general questions. First, is the
laboratory an acceptable complement to the field setting? That is, can
the laboratory setting be used to gain insight into or contribute to the
design of field instruments that would further the credibility of field
observations? Implicit in this view is the presumption that the
researcher has more Acontrol@ in the laboratory than in the field.
Second, can the laboratory be a substitute for field investigations in
contributing to policy decisions?[^4] That is, are there conditions
whereby the laboratory setting can be utilized in lieu of a field
setting for policy decisions?

Smith \[1988\] observed that studies that directly compare the outcome
of laboratory and field studies should be conducted to determine whether
student behavior reflects observed laboratory behavior. However, many
early studies of parallelism by Albert, \[1967\], and Hawkins et al. Al.
\[1977\], among others had concluded that student behavior could not be
used to predict the behavior of non-student decision-makers. More recent
literature suggests otherwise.[^5]

In market auction experiments, Dyer et al \[1989\] report that students
and business executives generated similar results in a common-value
experiment. Smith et al \[1988\] report that bubbles and crashes occur
with experienced traders and students. Using a principle-agent
experiment, DeJong et al \[1988\] demonstrate the average performance of
businessmen and student subject pools was the same, with the business
sample generating higher variance.

Addressing valuation issues, Brookshire et al \[1987a\] used a
non-student sample to compare laboratory versus field responses for a
market commodity and found that the values were the not substantially
different. Dickie et al \[1987\] found for the same commodity consistent
results for market and laboratory data. Brookshire and Coursey \[1987b\]
report consistent results for a public good between field and laboratory
settings. Neill et al \[1994\] compared the values elicited using an
Aindoor CVM@ for private goods with those obtained from Vickrey auctions
experiments for the same good. They found that the hypothetical values
from the CVM were much larger than from the auction and posited that the
differences are due to the hypothetical nature of the CV question.
Cummings et al \[1995 and 1997\] examine differences between
hypothetical and real CV questions for market and non-market goods,
respectively, and find higher valuations for the hypothetical CV
questions. Taking a different tack, Prince et al \[1992\] simulated a
field survey in the laboratory and concluded that an open-ended survey
design can yield unbiased estimates of WTP. They suggest laboratory
experiments can be used to inform field survey design.

The literature is replete with the arguments as to why the laboratory is
a Aproductive= setting for understanding field research. However, yet to
be addressed are the conceptual conditions necessary for the laboratory
investigations to be useful in helping to establish the credibility of
field non-market valuation. In general, the laboratory offers many
positive attributes through the use of a stylized setting that enables
the exploration of a multitude of institutions and treatments. These
settings allow the exploration of behavior for a multitude of treatments
at relatively low cost. Depending on the institutions chosen, the
Acorrect@ answer is known and thus the experiments can be designed to
explore the robustness of alternative information constructs, for
example.

The existing settings that link experiments with non-market valuation
for the most part have encompassed two types: experiments to examine the
formulation and statement of values (the WTA/WTP and hypothetical-bias
literature), and experiments designed to test for mechanism effects
(dichotomous choice vs. open-ended, money-back provisions, payment
vehicle, etc.)

The mechanism experiments have been very useful for providing an
inexpensive method for determining the implications of various survey
designs and sets of parameters. For the results of these experiments to
be applicable for extension to CV surveys, their design requires an
experimental setting that is very similar to that used in the field. In
a standard public goods experiment, individual subjects are asked to
contribute to the provision of a public good (essentially valuing their
potential expected dollar return from the good). They know their
incomes, the size of the population potentially contributing, and have a
subjective idea of possible returns (function of group size,
expectations of other=s contributions, and own payoff). Aside from the
good being valued (which is an actual dollar return), this would appear
to be very similar to the respondent=s task in a field CV survey.

However, we feel that there remain many unanswered questions. For
instance, what truly constitutes parallelism? That is, what are the
characteristics that are relevant and would contribute to our
understanding of the credibility of non-market valuations in field
settings. The extant literature has explored to some extent this
question, but no real assessment has been made along the lines that
benefit transfers guidelines underwent in previous years. We would argue
that in general, the result of the benefit transfer exercise yielded the
view that the process is fraught with more difficult issues than had
been originally envisioned. The same might be the case in setting out
the structure of parallelism.

If the laboratory setting is to have the potential to serve as a
complement or possibly a substitute for field efforts, a series of
systemic investigations need to be undertaken. For instance, what is the
outcome when two samples are utilized for a single institution and order
effects are taken into account? What is the outcome when different
information structures are utilized, such as the description of a Areal@
world public good (e.g. a public park) in lieu of the typical stylized
good?

Along these lines a related question would seem to be what is a proper
test of parallelism from the perspective of similarity of the
institution and application? That is, can there really be a complete
overlap between the laboratory and the field settings? Or will it always
be a leap? For instance, is the process compromised in that in the field
money typically does not change hands but in the laboratory money does
exchange hands? Must there be a complete one-to-one structural mapping
between the settings?[^6]

If a comparison of results between the laboratory and field for a public
good having induced private returns doesn=t hold up, then perhaps we
expect too much of the typical use of the CVM for valuing non-market
public goods. The argument being, if the process is not credible in the
simplest and most controlled setting how can one expect the outcome in a
more complex setting to be credible. Going further, if the laboratory
and field results are comparable for certain classes of goods such as
Acharismatic mega-fauna@ like endangered grizzlies and whales, why not
use the lab exclusively to elicit values for those types of goods?
Protocols for indoor CVM computer programs and subject recruiting could
be developed, and applied in labs across the country.[^7]

The process could be extended to real-time Internet public goods
provision experiments. We could develop Internet experiment/surveys that
could be played for several rounds.[^8] This melds the laboratory and
field setting in that it addresses the problem that traditional CVM is a
one-shot game. That is, the result we obtain from them includes only the
information (or lack thereof) that goes into the statement of one answer
to one question. If the individual doesn=t fully understand the
incentives that they face, how the mechanism works, the externalities in
gaming the system (over or under-revelation of their true value), they
don=t get a chance to observe the outcome and correct their answer.
Given the opportunity for repetition, we think CVM researchers might
obtain more accurate statements of values, as individuals see how their
valuations interact with others.

In sum, our focus on the issue of credibility has attempted to raise the
following questions:

1\) Why is the laboratory an interesting vehicle for insights to
non-market valuation?

2\) What issues does this literature raise and/or answer?

3\) What are the implications of this literature, either implied or
explicit for non-market valuation?

4\) What would be the precise structure of a replicable and transferable
design that would contribute to the credibility of field valuation
through the use of laboratory?

5\) Are there cases where laboratory results would suffice for policy
decisions? If so, what are the conditions? If not, why not?

References

Alpert, B., 1967, ANon-Businessmen as Surrogates for Businessmen in
Behavioral Experiments@, *Journal of Business* 40: 203-207.

Brookshire, D.S., and D. Coursey, 1987, AMeasuring the Value of a Public
Good: An Empirical Comparison of Elicitation Procedures.@ American
Economic Review 77(4): 554-556.

Brookshire, D.S., D. Coursey, and W. D. Schulze, 1987, AThe External
Validity of Experimental Economic Techniques: Analysis of Demand
Behavior.@ Economic Inquiry 25: 239-250.

Bagnoli, M., and M. McKee, 1991, AVoluntary Contribution Games:
Efficient Private Provision of Public Goods.@ *Economic Inquiry* 29 (2):
351-366.

Cummings, R. S., E. G. Harrison, and J. Murphy, 1997. AAre Hypothetical
Referenda Incentive Compatible?@ *Journal of Political Economy* 105(3):
609-621.

Cummings, R., G. Harrison, and E. Rutström, 1995, AHomegrown Values and
Hypothetical Surveys: is the Dichotomous Choice Approach Incentive
Compatible?@ *American Economic Review*, 85:260-266.

Dawes, Orbell, Simmons, and van de Kragt, 1986, AOrganizing Groups for
Collective Action.@ *American Political Science Review* 80: 1171-1185.

DeJong, D.V., R. Forsythe, and W.C. Uecker, 1988, AA Note on the Use of
Businessmen as Subjects in Sealed Offer Markets,@ *Journal of Economic
Behavior and Organization,* 9: 87-100.

Dickie, M., A. Fisher, and S. Gerking, 1987, A Market Transactions and
Hypothetical Demand Data: A Comparative Study@, Journal of the American
Statistical Association, 82: 67-75.

Dyer, D., J. Kagel, and D. Levin, 1989, AA Comparison of Naïve and
Experienced Bidders in Common Value Offer Auctions: A Laboratory
Analysis.@ *Economic Journal* 99: 108-115.

Hawkins, D., G. Albaum, and R. Best, AAn Investigation of Two Issues in
the Use of Students as Surrogates for Housewives in Consumer Behavior
Studies@, Journal of Business, 50: 216-223.

Neill, H., R. Cummings, P. Ganderton, G. Harrison, and T. McGuckin,
1994, AHypothetical Surveys and Real Economic Commitment@ *Land
Economics*, 70(2): 145-154.

Prince, R., M. McKee, S. Ben-David, and M. Bagnoli, 1992. AImproving the
Contingent Valuation Method: Implementing the Contribution Game*@
Journal of Environmental Economics and Management* 23: 78-90

Smith, V.L., G.L. Suchanek, and A.W. Williams, 1988, ABubbles, Crashes
and Endogenous Expectations in Experimental Spot Asset Markets@
Econometrica 56: 1119-1151.

Yves Van De Vyvere, 1994, AStated Preference Decompositional Modelling
and Residential Choice@, *Geoforum,* 25(2): 189-202.

[^1]: Prepared for the AAlternatives to Traditional CVM in Environmental
    Valuation@ NSF Workshop, Vanderbilt University, Nashville, TN.,
    October 15 and 16, 1998.

[^2]: Respectively, Chair and Professor of Economics, University of New
    Mexico and Assistant Professor of Economics, University of
    Tennessee.

[^3]: Parallelism and external validity carry different connotations. We
    will use the term parallelism for purposes of discussion in that we
    view it to be the broader term. Encompassed in parallelism are
    issues of the Asame answer in two settings@ as well as the
    transferability of results from one type of sample to another.

[^4]: This question in our mind is distinct from the question of
    calibration that has been raised by others.

[^5]: Our discussion of the literature is not meant to be comprehensive,
    just illustrative of some of the issues.

[^6]: Consider a stylistic example to illustrate some of our questions.
    Would a large-sample field CVM experiment that asked \$WTP for a
    share of a public good (a fund in dollars that returns total \$WTP/#
    respondents) be equivalent to a many observation laboratory
    experiment asking the same question?

[^7]: It is assumed the issue of sampling is appropriately addressed.

[^8]: It is our understanding that Trudy Cameron and possibly other are
    exploring this approach, as we are at UNM. This, however, raise
    issues of sampling.