1

A multi-level meta-analysis regression model of contingent

values for water ecosystem services

Abstract

This paper presents a meta-analysis of the contingent valuation literature on the value of water

services provided by lakes and rivers worldwide. We examine the service, site, socio-economic,

and study characteristics that determine public WTP for the water services provided by improved

water quality. This analysis represents a substantial step forward from previous meta-analyses on

this topic by utilising 3 to 4 times more value estimates from outside North America and by

using a GIS to augment the meta-data with spatially defined variables that represent the

availability of substitute sites and services. We find that the availability of substitutes in the form

of wetlands, as well as the relative abundance of lakes and rivers in the study areas, cause

estimated values to be lower. Using a multi-level modelling approach we find significant study

level differences in the value of the services related to water quality improvements, which

suggests the presence of authorship effects in the estimation of values.

Key words: Meta-analysis, Contingent Valuation, Water quality, Willingness to pay,

Substitution effect
2

1. Introduction

Fresh water ecosystems provide a wide range of ecosystem services that are economically highly

valuable, e.g. recreational opportunities, provision of drinking water, irrigation, habitat and

nursery for plants and animals, and may also have values that are not associated with any current

or future use (i.e. non-use values). The public good characteristics of many of these services

means that they are generally not traded in markets and as a consequence, freshwater ecosystems

are often undervalued in decisions related to their use, conservation or restoration. In response to

this lack of information on the value of goods and services from freshwater ecosystems, there is

now a sizable literature that attempts to estimate the values associated with improving or

maintaining the quality of lakes and rivers.

The economic value of changes in water quality has been the subject of several earlier metaanalyses (Johnston et al. 2003; 2005; 2006a; 2006b; Van Houtven et al., 2007; Moeltner et al.

2007; 2008; Stapler and Johnston, 2009). These previous studies have focussed on value

estimates from North America. The added value of the current paper is that is extends the data

both in terms of additional studies and value observations (also from Europe) and explanatory

variables, in particular spatially defined variables that capture the scarcity of freshwater,

wetlands and other substitute ecosystems in the vicinity of each study site. As such, it provides a

number of important new insights in the determinants of the value of the ecosystem services

provided by water quality improvements.

The structure of the paper is as follows. Section 2 provides a description of the contingent

valuation results that are used in this meta-analysis, including the location of study sites and the
3

definition of the dependent and explanatory variables; section 3 presents the estimated metaregression model, the results and interpretation; section 4 provides a discussion and conclusions.

2. Contingent valuation data on water quality in lakes and rivers

We collected and reviewed 154 contingent valuation studies that estimate values for ecosystem

services related to surface water quality. Of these studies, there are 48 that provide complete

information on all the explanatory variables that we include in the statistical meta-analysis (see

section 3). The most important missing information was a clear description of the change in

water quality that could be translated to a standardised measure. The standardised measure of

water quality that we use is described in detail below together with the other explanatory

variables. From the 48 selected studies we are able to code 349 separate value observations. This

is almost 3 times more than the number of observations in Van Houtven et al. (2007) and 4 times

the number of observations in Johnston and Thomassin (2010). We therefore obtain multiple

value observations from single studies, with an average of 7.3 observations per study. Separate

value observations from a study were included if they represent different study sites, sample

populations, ecosystem services, elicitation formats, or payment vehicles. These characteristics

of value observations are explicitly coded in the database and can be controlled for in the metaanalysis. It is likely, however, that value estimates from the same study are not independent and

we therefore test for the presence of hierarchical study effects in the data (see section 3).

The studies included in our analysis were published between the years 1981 and 2006. The

locations of study sites included in the database are largely in North America and Europe with a

small number in Central America, South East Asia, China and Australia. The geographic
4

distribution of study sites is presented in Figure 1. There is clearly an absence of available value

information for large areas of the world, particularly South America, Africa and most parts of

Asia.

Figure 1: Locations of water quality value study sites

Value estimates from contingent valuation studies are reported in a wide variety of currencies,

temporal units (e.g., per day, per visit, per month, present value etc.), and for different categories

and groupings of beneficiaries (e.g., per person, per household, per visitor etc.). We standardise

this value data to a common format, namely as willingness to pay (or accept) per household per

year for a change in water quality in USD 2007 price levels. The natural log of this value is the

dependent variable in the meta-analysis (VAL_N). Values that are originally reported as WTP

per person have been multiplied by the average household size for the country in which the study

is located. Values that are originally reported as WTP per other unit of time (e.g., day, month,

lump sum) have been converted to annual values using appropriate multipliers or by calculating

annualised values using time horizons and discount rates reported in the underlying study when
5

available. Values reported in other currencies or for price levels other than 2007 have been

converted to USD 2007 prices levels using PPP adjusted exchange rates and GDP deflators

obtained from the World Development Indicators (World Bank 2010). Value estimates of

willingness to accept (WTA) compensation are included for negative changes in water quality.

The explanatory variables used in the estimation of the value function include two measures of

water quality. These are the initial level of water quality (WQI_BASE) and the change in water

quality described in the contingent valuation scenario (WQI_CHG_ABS). Following Van

Houtven et al. (2007), we standardise the descriptions of water quality in the selected literature to

a 10-point index. This index is based on the water quality ladder developed by Vaughan (1986),

which translates descriptions of water quality in terms of associated suitability for a number of

activities into a 10-point index. For example, water quality that is considered “boatable” has a

water quality index value of 2.5, “fishable” water has a value of 5.1, and “swimmable” water has

an index value of 7.0.

The values for the majority of the explanatory variables are coded from information contained in

the primary valuation studies themselves. These include variables representing the type of

service valued, type of water body, survey method, payment vehicle, elicitation format, temporal

period of payment, and household income.

In addition we make use of a GIS to compute three spatially defined variables for each study site.

These are the area of lakes and rivers within a 10 km radius, the area of wetlands within a 50 km

radius, and the area of forests within a 50 km radius. These variables are intended to capture the
6

availability of substitute or complementary sites within the vicinity of each study site. The source

data for these variables is given in Table 2.

The dependent variable and the full set of explanatory variables used in the meta-regression are

listed in Table 1 together with the definition and descriptive statistics for each variable.
7

Table 1: Dependent and explanatory variables

Variable Variable definition Mean St. Dev.

VAL_LN WTP per household per year (USD; 2007 prices; natural log) 4.246 1.072

WQI_BASE Absolute change in 10-point water quality index 3.188 1.387

WQI_CHG_ABS Base level water quality in 10-point index 2.320 1.253

NON_USE Service: 1 = non-use value 0 = other 0.708 0.455

DRINK_QL Service: 1 = drinking water 0 = other 0.046 0.209

IRRIGATE Service: 1 = irrigation 0 = other 0.006 0.076

WATS_LAKE Water body: 1 = lake 0 = other 0.077 0.268

SUR_FACE Survey method: 1 = face-to-face 0 = other 0.576 0.495

SUR_MAIL Survey method: 1 = mail 0 = other 0.330 0.471

PAY_INCTX Payment vehicle: 1 = income tax 0 = other 0.135 0.342

PAY_TAX Payment vehicle: 1 = municipal tax 0 = other 0.115 0.319

ELC_DC Elicitation format: 1 = dichot. choice 0 = other 0.138 0.345

ELC_IB Elicitation format: 1 = iterative bidding 0 = other 0.390 0.488

ELC_PC Elicitation format: 1 = payment card 0 = other 0.163 0.370

TEMP_DAY Payment timing: 1 = day 0 = other 0.006 0.076

TEMP_TRIP Payment timing: 1 = trip 0 = other 0.023 0.150

TEMP_MON Payment timing: 1 = month 0 = other 0.138 0.345

HH_INC_LN Annual household income (natural log) 10.662 0.603

LAK10_LN Area of other lakes and rivers within 10km radius of study site

(hectares; natural log)

1.612 1.583

WET50_LN Area of wetlands within 50km radius of study site (hectares;

natural log)

4.583 2.865

FOR50_LN Area of forest within 50km radius of study site (hectares; natural

log)

7.853 1.214
8

Table 2: Spatial variables used in the meta-regression

Variable Description Source

LAK Area (ha) of rivers and

lakes within specified

radius of site

Global lakes and wetlands database GLWD. World

Wild Life – WWF and Center for Environmental

Systems Research, University of Kassel, Germany.

http://www.worldwildlife.org/science/data/item1877.

html

WET Area (ha) of wetlands

within specified radius

of site

Global lakes and wetlands database GLWD. World

Wild Life – WWF and Center for Environmental

Systems Research, University of Kassel, Germany.

http://www.worldwildlife.org/science/data/item1877.

html

FOR Area (km2) of forest

within specified radius

of site

The Global Land Cover Map for the Year 2000,

2003. GLC2000 database, European Commission

Joint Research Centre. http://wwwgem.jrc.it/glc2000.

3. Meta-regression model

The dependent variable in our regression equation is a vector of values in USD per hectare per

year in 2007 prices, labelled y. The explanatory variables are grouped in three different matrices

that include the site characteristics in

c

x (i.e., initial water quality, change in water quality, type

of ecosystem, services provided, and areas of substitute ecosystem sites), the study

characteristics in

a

x (i.e., survey method, payment vehicle, elicitation format, and temporal

period for payment), and the socio-economic characteristics of the study sample in

s

x (i.e., mean

household income).
9

The model fit was considerably improved by using the natural logarithm of the dependent

variable, household income, and the area of substitute ecosystem sites. Following Bateman and

Jones (2003) we use a multi-level modelling (MLM) approach to estimate the meta-regression.

1

The two-level MLM is similar to the one-way error components random effects model but allows

for more complex modelling of the variance components observed at each level of a specified

hierarchy. MLM allows a relaxation of the common assumption of independent observations,

and allows us to examine hierarchies within the data, such as similarity of estimates from the

same study. There are of course multiple possible sources of dependence in meta-analysis data

(see Rosenberger and Loomis, 2000). In the present study we examine dependence within studies

as in Johnston and Thomassin (2010) since we obtain a relatively large number of estimates per

study.

The use of MLM provides an indication of where the assumption of independence may be

invalid, and also improves the estimation of standard errors on parameter coefficients. The

practical advantage of using MLM over simply including study dummy variables is that it uses

data from all the observations in the sample to estimate the model. This pooling of information

results in ‘borrowing of strength’ Study specific relations, that would be poorly estimated if

modelled on their own, benefit from information from other studies (Orford, 2000). This results

in precision-weighted estimation, for which unreliable study specific implicit prices are

differentially shrunk towards the overall estimate. The estimated model is:

1 The software used is MLwiN version 2.0 (see Rasbash et al., 2003).
10

c c a a s s

ij ij ij ij j ij

y = α +  X +  X +  X + μ + ε ,

where the subscript i takes values from 1 to the number of observations and subscript j takes

values from 1 to the number of studies, α is the constant term, μjis an error term at the second

(study) level,

ij

ε is an error term at the first (observation) level, and the vectors

c

 ,

a

 and

s



contain coefficients to be estimated by the model on explanatory variables in X

c

, X

a

and X

s

,

respectively. We assume that μj and ij

ε follow a normal distribution with means equal to zero

and that they are uncorrelated, so that it is sufficient to estimate their variances,

2 σμ

and

2 σε

respectively. This type of model is also known as a variance components model, given that the

error variance is partitioned into components corresponding to each level in the hierarchy. In our

model, the level 2 error term represents each study’s departure from the population mean,

represented by the constant term.

In order to control for differences in the precision of the underlying value estimates, we weight

the observed values by the square root of the sample size in the underlying study, as proposed by

Hunter and Schmidt (1990). This weighting procedure has been shown to be less efficient but

also less biased than using weights based on the standard errors of the effect sizes (SanchezMeca and Marin-Martinez, 1998). Standard errors for the value estimates in our sample are,

however, generally not always reported and so this alternative weighting option was not possible.

The effect of weighting the observed values does not have a substantial effect on the estimated

coefficients, but does drawn in the standard errors. The meta-regression results are presented in

Table 3.
11

The overall model fit is good. The McFadden psuedo R

2

statistic lacks the straightforward

interpretation of the standard OLS R

2

, but indicates that a high proportion of variation in the

dependent variable is explained by variation in the explanatory variables included in the model.

While controlling for several important ecosystem service, site, population and study

characteristics, we still find significant study level differences in value estimates for water

quality improvements as indicated by the level 2 variance. The variance partition coefficient

indicates that 51% of the total unexplained variance in the value of open space may be attributed

to differences between studies. This suggests that there are important author or study effects that

influence estimated values for water quality improvements.

Turning to the coefficient estimates, the estimated coefficient on the initial water quality variable

(WQI_BASE) is - as in Johnston and Thomassin (2010) - negative and statistically significant at

the 5% level, indicating that WTP for water quality improvement is lower when water quality is

already high. The corollary is that the public is willing to pay more to improve bad water quality.

The coefficient on the water quality change variable (WQI_CHG_ABS) is positive and

statistically significant at the 10% level just as in Van Houtven et al. (2007), indicating that the

public are sensitive to scope and are willing to pay more for larger changes in water quality.

New in the analysis presented here compared to the existing meta-analyses is the inclusion of a

set of binary variables indicating the ecosystem services valued. We find that value estimates for

cultural services that include non-use values such as the protection of wildlife habitat and

biodiversity conservation tend to be higher than direct use recreational services (the omitted

category). Provisioning services such as irrigation and drinking water quality do not have
12

statistically significantly different values from the omitted category. These results suggest that

non-use values may comprise a substantial share of total value for water quality improvements.

Contrary to the findings in Johnston and Thomassin (2010), the estimated coefficient on the

dummy variable indicating that the valued water body is a lake is negative and statistically

significant at the 5% level. This shows that water quality changes in lakes are of lower value than

similar changes in rivers.

The face-to-face and mail survey formats are found to result in lower elicited values than

telephone surveys (the omitted category). This too is different from the findings in Van Houtven

et al. (2007) and Johnston and Thomassin (2010), who both find a significant positive effect of

in-person interviews on stated WTP values. Contrary to the findings in Brouwer et al. (1999),

using income tax as the payment vehicle results in lower values than other payment vehicles. The

use of municipal taxes as the payment vehicle, however, is found to result in statistically

significantly higher values. The elicitation format also has a statistically significant influence on

estimated WTP, with the dichotomous choice, iterative bidding and payment card formats all

producing, as expected from the existing literature (e.g. Bateman et al., 1995; Brouwer et al.,

1999), higher values than open ended formats (the omitted category).
13

Table 3: Meta-regression results

Variable Variable definition Coefficient Std. Error

Constant 1.759 1.202

WQI_BASE Absolute change in 10-point water quality index -0.148** 0.060

WQI_CHG_ABS Base level water quality in 10-point index 0.116* 0.061

NON_USE Service: 1 = non-use value 0 = other 0.543** 0.242

DRINK_QL Service: 1 = drinking water 0 = other 0.301 0.416

IRRIGATE Service: 1 = irrigation 0 = other -0.337 0.473

WATS_LAKE Water body: 1 = lake 0 = other -0.743** 0.350

SUR_FACE Survey method: 1 = face-to-face 0 = other -0.923*** 0.298

SUR_MAIL Survey method: 1 = mail 0 = other -0.558** 0.251

PAY_INCTX Payment vehicle: 1 = income tax 0 = other -0.995*** 0.344

PAY_TAX Payment vehicle: 1 = municipal tax 0 = other 0.468* 0.279

ELC_DC Elicitation format: 1 = dichotomous choice 0 = other 0.839*** 0.196

ELC_IB Elicitation format: 1 = iterative bidding 0 = other 0.182** 0.089

ELC_PC Elicitation format: 1 = payment card 0 = other 0.505*** 0.170

TEMP_TRIP Payment timing: 1 = trip 0 = other 1.812*** 0.314

TEMP_DAY Payment timing: 1 = day 0 = other 0.818*** 0.307

TEMP_MON Payment timing: 1 = month 0 = other 0.892*** 0.297

HH_INC_LN Annual household income (natural log) 0.293*** 0.112

LAK10_LN Area of other lakes and rivers within 10km radius of study site -0.082** 0.041

WET50_LN Area of wetlands within 50km radius of study site -0.063* 0.038

FOR50_LN Area of forest within 50km radius of study site 0.009 0.061

Level 1 (estimate) variance 0.327*** 0.052

Level 2 (study) variance 0.345*** 0.078

–2*loglikelihood 682.851

Psuedo R

2

0.368

N 349

***

,

**

,

*

= statistically significant at 1%, 5%, and 10%, respectively.
14

The temporal period for which payments are made (as described in the hypothetical payment

scenario of the CV study) has a strong influence on the WTP elicited. Shorter time periods (visit,

day, month) are, as expected from the existing literature (e.g. Loomis and White, 1996), shown

to result in statistically significantly higher WTP than scenarios that used annual payments (the

omitted category).

The positive sign on the income variable indicates that WTP increases as expected with income.

Given the double log specification of the regression model, the coefficient in this case can be

interpreted as the elasticity of WTP with respect to income. In other words, water quality is a

normal good. WTP for water quality improvement is found to be positively related but inelastic

with respect to income.

The estimated coefficient on the variable measuring the area of lakes and rivers in the vicinity of

each study site is negative and statistically significant at the 5% level, indicating that respondents

are willing to pay less for water quality improvements in a specific water body if there are more

other lakes and rivers nearby. This result suggests that respondents are sensitive to the

availability of substitute sites. Similarly, we obtain a negative and statistically significant

coefficient on the abundance of wetlands in the vicinity of the study site. This implies that to

some extent respondents perceive wetlands to be substitutes for lakes and rivers, and are

therefore willing to pay less for water quality improvements in a water body if there is an

abundance of wetlands in the surrounding landscape. The estimated effect of forest abundance on

the other hand is positive, albeit not statistically significant.
15

4. Discussion and conclusions

Several stated preference meta-analyses have been conducted in the past years related to fresh

surface water. However, these studies focus on North American valuation studies only.

Furthermore, this existing literature does not systematically examine the various goods and

services provided by aquatic ecosystems. The meta-analysis presented here is based on a

substantially extended data base including besides the water valuation studies from North

America also the growing stock of European, Asian and Australian stated preference water

valuation studies. As expected, the nature of the water services, the characteristics of the sites

where these services are provided (type of water body, baseline quality level and the change in

quality level), the socio-economic characteristics of the population of beneficiaries (their ability

to pay) and the characteristics of the methodological approach (survey administration, payment

vehicle, WTP elicitation procedure) all influence stated WTP in the existing contingent valuation

literature.The work presented in this conference paper is in progress. A more substantial

comparison of our meta-regression results and the existing meta-analyses in this field will be

carried out to extend the paper. Results so far show that we find similarities, but also striking

unexpected dissimilarities compared to the existing North American meta-analysis studies.

Further analysis will be conducted to explore the underlying determinants of these differences .

To this end, the same regression functions will be estimated as in the published studies based on

an extended number of observations from Europe, Asia and Australia, to test for structural

differences in values for surface water across regions. This analysis will also allow us to test the

effect of the inclusion of additional contextual variables.Besides the increase in the sheer number

of observations, we also include important additional spatial study characteristics in the

regression model, most notably the effect of the abundance of similar water and wetland
16

resources and other ecosystems providing similar public recreational and cultural services. We

show that studies conducted in areas where there are more lakes, rivers and wetlands produce

significantly lower WTP values. These variables are rough indicators of substitutability. Future

research in this area will also try to account for the quality of the available water resources (and

provided services) in the study areas as this is expected to provide a more reliable and robust

indicator of substitution effects.

Finally, another next step in the analysis presented here is to test the performance of in and out of

sample transfers based on the estimated meta-regression model. This implies that additional

control factors will be included for the countries and regions in which the valuation studies were

conducted. Substantial differences exist between countries in terms of value-data density.

Important questions are to what extent public WTP values can be reliably predicted in a data

poor country using a meta-regression model based on the valuation results from a data rich

country. Johnston and Thomassin (2010) point out the risk involved in international site-to-site

transfers that do not capitalize on the insights available through meta-analysis. We therefore aim

to further test the effect of utilising the meta-regression model results in international value

transfers for policy analysis.
17

References

Bateman, I. J., Jones, A.P., 2003. Contrasting Conventional with Multi-Level Modeling

Approaches to Meta-Analysis: Expectation Consistency in U.K. Woodland Recreation Values.

Land Economics. 79, 235–258.

Brouwer, R., Langford, I.H., Bateman, I.J. & Turner, R.K. (1999). A meta-analysis of wetland

contingent valuation studies. Regional Environmental Change, 1(1), 47-57.

Hunter, J.E., Schmidt, F.L., 1990. Methods of Meta-Analysis: Correcting Error and Bias in

Research Findings. London: Sage Publications.

Johnston, R.J., Besedin E.Y., and Wardwell R.F., (2003). Modeling Relationships between Use

and Nonuse Values for Surface Water Quality: A Meta-Analysis. Water Resources Research

39(12): 1363-72.

Johnston, R.J., Besedin, E.Y., Iovanna, R., Miller, C.J., Wardwell, R.F. and Ranson, M.H.

(2005). Systematic variation in willingness to pay for aquatic resource improvements and

implications for benefit transfer: A meta-analysis, Canadian Journal of Agricultural Economics

53: 221-48.

Johnston, R.J., Ranson, M.H., Besedin, E.Y. and Helm, E.C. (2006a). What determines

willingness to pay per fish? A meta-analysis of recreational fishing values, Marine Resource

Economics 21: 1-32.
18

Johnston, R.J., Besedin, E.Y., and Ranson, M.H., (2006b). Characterizing the effects of valuation

methodology in function-based benefits transfer. Ecological Economics, 60: 40 7-419.

Johnston, R.J. and Thomassin, P.J. (2010). Willingness to pay for water quality improvements in

the United States and Canada: Considering possibilities for international meta-analysis and

benefits transfer. Agricultural and Resource Economics Review, 39(1): 114-131.

Loomis, J.B., White, D.S., 1996. Economic benefits of rare and endangered species: summary

and meta-analysis. Ecological Economics 18, 197–206.

Moeltner, K., Boyle, K.J. and Paterson, R.W. (2007). Meta-analysis and benefit transfer for

resource valuation–Addressing classical challenges with Bayesian modeling, Journal of

Environmental Economics and Management 53: 250-69.

Moeltner, K. and Rosenberger, R.S., (2008). Predicting Resource Policy Outcomes via MetaRegression: Data Space, Model Space, and the Quest for ‘Optimal Scope’. The B.E. Journal of

Economic Analysis & Policy, 8.

Orford, S., 2000. Modelling spatial structures in local housing market dynamics: A multi-level

perspecitive. Urban Studies, 37: 1643-1671.
19

Rosenberger, R.S., Loomis, J.B., 2000. Panel Stratification in Meta-Analysis of Economic

Studies: An Investigation of Its Effects in the Recreation Valuation Literature. Journal of

Agricultural and Applied Economics. 32, 459–470.

Sanchez-Meca, J., Marin-Martinez, F., 1998. Weighting by Inverse Variance or by Sample Size

in Meta-analysis: A Simulation Study. Educational and Psychological Measurement. 58, 211–

220.

Stapler, R.W., and Johnston, R.J. (2009). Meta-Analysis, Benefit Transfer, and Methodological

Covariates: Implications for Transfer Error. Environmental and Resource Economics, 42: 227-

246.

Van Houtven, G., Powers, J., and Pattanayak S.K. (2007). Valuing water quality improvements

in the United States using meta-analysis: Is the glass half-full of half- empty for national policy

analysis? Resource and Energy Economics. 29: 206-228.

Vaughan, W.J. (1986). The resource for the future water quality ladder, Appendix B in Mitchell,

R.C. and Carson, R.T., The use of contingent valuation data for benefit-cost analysis in water

pollution control, final report. Washington, D.C.: Resource For The Future.

World Bank (2010). World Development Indicators.