1

Date August 19, 2015

To Todd Doley, Julie Hewitt, and James Covington, U.S. Environmental Protection 

Agency

From Matthew Ranson and Elena Besedin, Abt Associates

Subject Revised Memorandum: Accounting for Scope in the MRM2 Meta-Regression Model

This memorandum addresses the issue of whether and how the meta-model should account for the 

magnitude of the water quality changes presented to survey respondents in stated preference surveys. In 

MRM1, water quality change is the key explanatory variable—and it is highly statistically significant. 

However, as described articulately in NCEE’s 4/23/2015 memorandum, the use of water quality change in 

a log specification results in a valuation function that does not satisfy the adding-up property.

Overall, we understand and generally agree with NCEE’s desire to develop a meta-model that satisfies 

that adding-up property. Yet, economists have long recognized that survey respondents often express 

preferences that appear inconsistent with this property (Diamond et al, 1993). Consistent with this 

general finding, our raw meta-data also suggest that survey respondents express high marginal 

willingness-to-pay (WTP) for modest improvements in water quality, but then express much lower 

marginal WTP for additional improvements—regardless of their current baseline level of water quality. 

There are a number of potential explanations for this pattern, including rational explanations such as 

income effects; survey design issues such as capturing warm glow effects or choosing water quality 

change scenarios that respondents have difficulty visualizing or interpret as being unrealistic; and actual 

inconsistencies in respondents’ preferences.

Regardless of the cause of this pattern, developing a meta-model that satisfies adding-up requires a 

strategy for imposing the adding-up property on the meta-data. The approach currently used by MRM2 is 

to exclude water quality change from the MRM2 regression equation. In other words, MRM2 calculates 

marginal WTP values for a particular water quality level by averaging all relevant observations in the 

dataset, regardless of whether they are derived from surveys that asked respondents to value large, 

medium, or small water quality changes around that level. Effectively, this assumes that survey 

respondents’ preferences are best represented by their stated WTP for water quality improvements that are 

average in scope in the underlying meta-data. 

The purpose of this memo is to propose an approach for augmenting MRM2 by conducting a sensitivity

analysis to illustrate how marginal WTP is influenced by the scope of the water quality change scenarios 

valued in different stated preference studies. In particular, we propose estimating a modified version of 

MRM2 in which water quality change is treated as a study methodological characteristic. In other words, 

we suggest including water quality change as a demand shifter variable that affects marginal WTP in the 

Memorandum
2

same way as other study methodological characteristics (e.g., the use of a choice experiment format, or a 

lump sum payment). Like those parameters, the water quality change parameter could be set to any 

desired value, based on the scope of water quality changes (small, medium, large) that a well-designed 

stated preference survey would use if it were eliciting marginal WTP for the types of improvements 

expected under future Clean Water Act regulations. Importantly, because water quality change would be 

treated as a methodological characteristic instead of as a scenario characteristic, the value assigned to the 

water quality change parameter need not correspond to the water quality change being valued. For the 

same reason, the resulting model would completely satisfy the adding-up condition, as long as the water 

quality change parameter were held constant across benefit transfer applications for different future 

regulatory analyses.

We believe that this approach has three strengths. First, it allows the meta-model to account for water 

quality change, which as a purely empirical matter is a strong predictor of WTP.

1

 Second, this approach 

provides a transparent way of conducting an uncertainty analysis that explores how scope affects marginal 

WTP. Third, the resulting model satisfies the adding-up condition for a given value of the water quality 

parameter.

In summary, the overall purpose of the meta-analysis is to predict the WTP values that would be 

generated by conducting a stated preference survey with a particular set of methodological, 

environmental, and participant characteristics. Including water quality change as a methodological 

variable in MRM2 would allow EPA to understand how the framing of a stated preference survey—in 

particular, the magnitude of the size of the water quality change considered—affects the marginal WTP 

values implied by respondents’ stated preferences.

The remainder of this memorandum is organized as follows. Section 1 describes MRM2 in detail, in 

order to ensure that we share a common understanding of this model with NCEE. Section 2 then 

discusses the functional form of MRM2, and shows that it does not capture the strong relationship 

between WTP and water quality change that is apparent in the meta-dataset. Finally, Section 3 presents 

our proposed approach for incorporating water quality change as a methodological parameter in MRM2, 

and illustrates how it could be used to estimate benefits from the Steam Electric Regulation.

 

1

For example, including the log of water quality change as an additional variable in MRM2 increases the R2of the 

model from 0.61 to 0.72. This increase in explanatory power from the addition of a single variable is particularly 

striking, given that MRM2 already includes 23 other variables and a constant.
3

1. Estimating Willingness-to-Pay Using MRM2

This section provides a detailed discussion of the approach that MRM2 uses to estimate willingness-topay for water quality changes. The purpose of this discussion is to ensure that our understanding of 

MRM2 is correct, and to motivate the alternative modeling approach we present in Section 3 below.

In its most general form, MRM2 models marginal willingness-to-pay, MWTPi ≡

∂WTPi

∂Q

, for observation i, 

as follows:

ln(MWTPi

) = α0 + α1Xi + f(Qi

; β) + εi(1)

In this equation, α0 is a constant, Xiis a vector of study and resource characteristics that act as demand

shifters, Qiis the absolute water quality index level at which marginal willingness-to-pay is being 

evaluated, and εiis a zero-mean normally-distributed error term. The function f(⋅) expresses the core 

relationship between absolute water quality Qi and marginal WTP, and β is a vector of parameters that 

define the functional form of f(⋅).

The key question of interest is what is an appropriate form for f(⋅). NCEE’s 4/23/2015 memorandum 

assumes a reasonable and parsimonious functional form that is linear in Qi:

f(Qi; β) ≡ β ⋅ Qi(2)

This leads to the following equation for marginal willingness-to-pay:

ln(MWTPi

) = α0 + α1Xi + β ⋅ Qi + εi

(3)

which corresponds to Equation (7) in NCEE’s 4/23/2015 memorandum.

However, one remaining empirical challenge is that MWTPiis not observed directly in the meta-data, and 

instead must be calculated from total willingness-to-pay as part of the estimation routine. This would 

require a nonlinear least squares or maximum likelihood estimation approach. As a simpler alternative, 

NCEE’s 4/23/2015 memorandum assumes that marginal WTP can be approximated by average WTP per 

unit of water quality:

MWTPi ≈

WTPi

ΔQi

(4)

with this approximation assumed to be valid at the midpoint of the water quality change valued for that 

meta-data observation: 

Qi =

Qi0+Qi1

2

(5)

Substituting Equations (4) and (5) into Equation (3) provides the version of MRM2 that is ultimately used 

as a regression model: 
4

ln (

WTPi

ΔQi

) = α0 + α1Xi + β ⋅ (

Qi0+Qi1

2

) + εi(6)

The simplest version of this regression uses an unweighted OLS approach to generate a vector of 

estimated parameter values (α̂0,α̂1, β̂, σ̂

2

), where σ̂

2

is the sum of squared residuals from equation (6). 

MRM2 then substitutes these estimated values into Equation (3) above, to get an expression that can be 

used to predict the log of marginal willingness-to-pay:

ln(MWTPi

) = α̂0 + α̂1Xi + β̂ ⋅ Qi + εi

(7)

Taking the exponent of both sides produces the following expression for marginal WTP evaluated for a 

particular set of study/resource characteristics X and a particular water quality index level Q:

MWTP = exp(α̂0 + α̂1X + σ̂

2

/2) ⋅ exp(β̂ ⋅ Q) (8)

Small changes in water quality (e.g., less than one unit) could be valued using an approximation that 

involves multiplying predicted MWTP from equation (8) by the amount of the water quality change.2 

Larger water quality changes must account for the curvature of the marginal WTP function, and so must 

be calculated using the integral:

WTP = ∫ exp(α̂0 + α̂1X + σ̂

2

/2) ⋅ exp(β̂ ⋅ Q)

Q1

Q=Q0

dQ (9)

which is equal to:

WTP = exp(α̂0 + α̂1X + σ̂

2

/2) ⋅

exp(β̂⋅Q1)−exp(β̂⋅Q0)

β̂

(10)

As noted in NCEE’s 4/23/2015 memorandum, calculating total WTP using this approach has the key 

strength of providing a valuation function that is consistent with the adding-up property.

 

2 We note that 98% of reach miles affected by the Steam Electric Regulation would experience WQI changes that 

range between 0 and 1.
5

2. Marginal WTP Depends Strongly on Water Quality Change

The previous section describes the functional form of the MRM2 model. In this section, we show that 

this model’s functional form is not completely consistent with some underlying empirical patterns of the 

meta-dataset.

A key feature of MRM2, as described in its general version in Equation (1) above, is that marginal WTP 

does not depend on the magnitude of the water quality change being valued. In other words, MRM2 

assumes that marginal WTP for the one-unit change from 49 to 50 on the water quality index is the same, 

regardless of whether survey respondents live in an area where baseline water quality is 35 or 49, and 

whether the expected total water quality change is 30 points or 2 points.

As an empirical matter, the meta-data do not support this assumption. Instead, they suggest that marginal 

WTP is a nonlinear, decreasing function of water quality change. More specifically, the meta-data show

that survey respondents express high marginal WTP for modest improvements in water quality, but then 

express much lower marginal willingness to pay for additional improvements.

To demonstrate this point, Figure 1 graphs total WTP as a function of water quality change, for meta-data 

observations falling into four different baseline water quality levels: 0-19, 20-39, 40-59, and 60-80. 

(Note that there are no meta-data observations with baseline water quality levels greater than 80.) This 

figure is reproduced from our hybrid model memorandum. Although there is considerable noise in the 

figure, these panels demonstrate that total WTP increases quickly for the first 10 to 20 units of additional 

water quality, and then plateaus. This implies that marginal WTP is large and positive for the first 10 to 

20 units of additional water quality, and then declines toward zero. This pattern is roughly consistent 

across all four panels, indicating that this stylized fact appears to hold true regardless of whether baseline 

water quality is low or high.
6

Figure 1. WTP by Water Quality Change and Baseline Water Quality

(a) Baseline WQ: 0-19 (b) Baseline WQ: 20-39

(c) Baseline WQ: 40-59 (d) Baseline WQ: 60-80

Notes: Each panel shows the expected value and 95% confidence interval for mean WTP based on the 

meta-data, by 10-unit water quality change bin. The dots in each panel show the raw meta-data values.
7

3. Accounting for Water Quality Change in the Meta-Model

The previous section shows that the magnitude of water quality change has a strong influence on survey 

respondents’ valuations. Given this is an empirical feature of the meta-dataset, what is the appropriate 

way to develop a meta-model for predicting marginal willingness-to-pay? 

3.1.1 Treating Water Quality Change as a Study Methodological Characteristic

We propose addressing this issue by treating water quality change as a study methodological 

characteristic. In other words, we propose modeling water quality change as a demand shifter that affects 

marginal WTP in the same way as other study methodological characteristics included in the meta-model 

(e.g., the use of a choice experiment format, or a lump sum payment). And, like those parameters, the 

water quality change parameter could be set to any appropriate value when the meta-model is used for 

benefit transfer. Importantly, because water quality change would be treated as a methodological 

characteristic instead of as a scenario characteristic, the value assigned to the water quality change 

parameter need not correspond to the specific water quality change being valued. For the same reason, 

the resulting model will still satisfy the adding-up property. 

As noted above, the approach adopted by MRM2 is to model marginal WTP using the average water 

quality change magnitude in the meta-dataset.3 In other words, MRM2 assumes that the average water 

quality change scenario in the meta-dataset is what a well-designed stated preference survey would use 

for eliciting respondents’ marginal WTP. While this approach is reasonable, other strategies are also 

possible.

A hypothetical example may help to illustrate this point. Suppose that a particular contingent valuation 

study elicits WTP from respondents in two counties: County 1 and County 2. Respondents in County 1 

have an average WTP of $30 for a +2 unit improvement from their current baseline of 64 to a future value 

of 66 (on the 100-point Water Quality Index). Respondents in County 2 have an average WTP of $50 for 

a +10 unit improvement from their current baseline of 60 to a future value of 70. These results imply that 

marginal WTP for the 65th unit of water quality is approximately $15 in County 1, and $5 in County 2.4 

These valuations clearly do not satisfy adding-up, since the marginal value of water quality changes 

depends on the scope of the change being valued.

Nonetheless, the authors would like to use the results of their contingent valuation study to value the 

benefits of a small water quality improvement in a third county (County 3). County 3 is very similar to 

County 1 and 2, except that baseline water quality is 64.5. The question the authors face is: what 

marginal willingness-to-pay value should they use to value a one-unit water quality increase in County 3? 

Unfortunately, there is no clear theoretical guidance for this decision. Values of $5 or $15 are certainly 

possibilities, as is the average value of $10. And in some sense, all of these values are arbitrary, since the 

study might have gotten different marginal WTP estimates had it asked respondents to value water quality 

changes of different magnitudes. Thus, the most appropriate course of action might be to conduct a 

 

3 Note that this statement is an approximation, since MRM2 uses a regression model (not an average) to calculate 

marginal WTP. However, the basic intuition is correct.

4 These hypothetical marginal willingness-to-pay calculations use the same midpoint approximation approach as 

MRM2.
8

sensitivity analysis using marginal values of both $5 and $15. In effect, this would treat water quality 

change as a study methodological characteristic, and estimate marginal WTP using the range of water 

quality change values found this particular survey. We believe that an identical intuition applies to EPA’s 

meta-analysis.

3.1.2 Modeling Details

To formalize the alternative approach we are proposing, the following equation presents an expanded 

version of MRM2 that allows the water quality change ΔQito enter as a methodological characteristic. 

We refer to this model as MRM2-S (where the “S” indicates that the model takes into account the scope 

of the water quality change):

ln(MWTPi

) = α0 + α1Xi + g(ΔQi

; φ) + f(Qi; β) + εi(11)

In this equation, ΔQiis the magnitude of the water quality change for meta-data observation i, and φ is a 

vector of parameters that specify the functional form for the function g(⋅). For simplicity, we model the 

function g(⋅) as a linear function of the water quality change:

5

g(ΔQi; φ) ≡ φ ⋅ ΔQi(12)

Substituting Equations (4), (5), and (12) into Equation (11) gives the following regression model:

ln (

WTPi

ΔQi

) = α0 + α1Xi + φ ⋅ ΔQi + β ⋅ (

Qi0+Qi1

2

) + εi(13)

Following the same logic described at the end of Section 1, the estimated parameters from this equation 

can be used to generate an expression for both marginal WTP and total WTP:

MWTP = exp(α̂0 + α̂1X + φ ⋅ ΔQ + σ̂

2

/2) ⋅ exp(β̂ ⋅ Q) (14)

WTP = exp(α̂0 + α̂1X + φ ⋅ ΔQ + σ̂

2

/2) ⋅

exp(β̂⋅Q1)−exp(β̂⋅Q0)

β̂

(15)

Again, we emphasize that ΔQ is treated as a study methodological characteristic, not as a characteristic of 

the benefit transfer water quality improvement scenario. Thus, in equations (14) and (15), the parameter 

ΔQ need not be set equal to Q1 − Q0.

3.1.3 Regression Results

Table 1 presents regression results for two versions of MRM2-S. Column (b) shows the linear version, 

exactly as described in Equation (13). Column (c) presents an alternative version, which instead includes 

the natural log of the change water quality. As a reference, Column (a) presents regression results for the 

main MRM2 model. 

 

5 We also consider a specification in which we model g(⋅) as a linear function of the natural log of water quality 

change.
9

In general, the coefficients on the methodological, socioeconomic, and resource characteristic control 

variables are quite similar across the three models. There are some differences in magnitude and 

significance, but the signs of all variables remain consistent. 

Focusing on the absolute water quality variable (Qavg2) shows that water quality has a larger effect on 

marginal WTP under MRM2, compared to the MRM2-S models. The coefficient is -0.017 (p<.01) under 

MRM2, -0.012 (p<.05) under the linear water quality change MRM2-S model, and -0.008 (ns) under the 

log water quality change MRM2-S model. This pattern suggests that after accounting for the change in 

water quality, absolute water quality may have less of an effect on marginal willingness to pay. However, 

we do note that a t-test would fail to reject the equality of these coefficients.

The MRM2-S models also include water quality change variables: either quality_ch or lnquality_ch. 

Columns (b) and (c) show that in both models, these variables are negative and highly significant. This 

pattern indicates that marginal WTP is decreasing in the size of the water quality change being valued. 

This is consistent with the patterns from the raw meta-data, as discussed in Section 2 above.

Overall, a comparison of the R2values from these three models indicates MRM2-S has greater 

explanatory power than MRM2. MRM2 has a R2value of 0.612. In contrast, the R2value for the linear 

MRM2-S model is 0.677, and the R2value for the log MRM2-S model is 0.722. Given that MRM2 

already has 23 variables and a constant, this substantial increase in R2from one additional variable 

suggests that water quality change (or log water quality change) is an important variable to include in the 

model.
10

Table 1. Regression Results for MRM2 with Water Quality Change Included

(a) (b) (c)

Parameter

MRM2 MRM2-S:

Water Quality Change

MRM2-S:

Log Water Quality Change

ce 0.530 (0.308) 0.503* (0.241) 0.512* (0.199)

thesis 0.631* (0.246) 0.723** (0.196) 0.702** (0.184)

lnyear -0.356** (0.101) -0.404** (0.089) -0.452** (0.080)

volunt -1.468** (0.225) -1.337** (0.219) -1.272** (0.206)

outlier_bids -0.494** (0.175) -0.380* (0.150) -0.391** (0.122)

nonparam -0.577** (0.119) -0.478** (0.111) -0.503** (0.110)

non_reviewed -0.578** (0.200) -0.638** (0.177) -0.686** (0.165)

lump_sum 0.879** (0.190) 0.809** (0.154) 0.786** (0.141)

WTP_median -0.348 (0.342) -0.305 (0.242) -0.265 (0.223)

northeast 1.058** (0.321) 0.846** (0.271) 0.593* (0.254)

central 0.545** (0.161) 0.621** (0.130) 0.611** (0.113)

south 1.351** (0.171) 1.427** (0.152) 1.429** (0.138)

nonusers -0.454** (0.151) -0.428** (0.124) -0.448** (0.122)

lnincome1 0.212 (0.420) 0.461 (0.370) 0.638 (0.325)

mult_bod -0.680** (0.205) -0.599** (0.151) -0.527** (0.142)

river1 -0.092 (0.151) -0.136 (0.113) -0.230* (0.111)

swim_use -0.384* (0.184) -0.307 (0.172) -0.347* (0.172)

gamefish 0.526 (0.308) 0.483 (0.241) 0.290 (0.179)

boat_use -0.715** (0.226) -0.500** (0.181) -0.382* (0.156)

ln_ar_agr -0.344** (0.100) -0.361** (0.102) -0.387** (0.088)

ln_ar_ratio1 -0.075** (0.026) -0.070* (0.028) -0.071** (0.025)

sub_frac 1.135** (0.237) 0.908** (0.211) 0.714** (0.182)

Qavg2 -0.017** (0.005) -0.012* (0.005) -0.008 (0.005)

quality_ch -0.027** (0.008) 

lnquality_ch -0.664** (0.119)

_cons 0.150 (4.611) -2.172 (4.051) -2.725 (3.484)

Observations: 140 140 140

R

2

0.612 0.677 0.722

Weights No No No

Random Effects: No No No

Notes: The table shows coefficients for each model, with standard errors in parentheses. * denotes p<.05; ** 

denotes p<.01. Due to time constraints, the standard errors in the table are taken directly from the OLS 

regressions, and do not use the bootstrapping approach described in NCEE’s 4/23/2015 memorandum.

3.1.4 Determining an Appropriate Value for the Water Quality Change Parameter in MRM2-S

If the MRM2-S model were used to estimate the benefits of the Steam Electric Rule (or other future EPA 

regulations), EPA would need to determine an appropriate value for the water quality change parameter. 

This section discusses some considerations related to that choice.

The key underlying question is: why does marginal WTP depend on the scope of the water quality 

change being valued? There are several potential hypotheses that could explain why survey respondents’ 

implied marginal WTP appears to depend on baseline water quality conditions in the area in which they 

live. 

 One hypothesis is that this pattern simply reflects an income effect.
11

 A second hypothesis is that survey respondents may have difficulty visualizing very large or very 

small water quality changes. This could lead them to understate their willingness-to-pay for large 

changes, and to overstate their willingness-to-pay for small changes.

 A third hypothesis is that survey respondents may find large water quality changes implausible. 

As a result, they might discount their stated valuations for these large changes to account for what 

they perceive as the low probability of the changes actually occurring.

 A fourth hypothesis is that stated preference surveys capture “warm glow” effects (Diamond et al, 

1993). In other words, survey respondents may provide WTP values that reflect their desire for 

action, or their utility from doing good deeds. Under this hypothesis, much or all of their 

monetary WTP would reflect this general warm glow, instead of their actual valuation for the 

specific resource change under consideration. Note, however, that researchers have had some 

success using cheap talk scripts, certainty adjustments, and other methods reduce and minimize

hypothetical biases (e.g., Champ et al. 1997, 2004; Cummings and Taylor 1999; Johnston 2006; 

Ready et al. 2010). Recently published findings of Ready et al. (2010, p. 363), for example, show 

that contemporary certainty adjustments “successfully mitigated hypothetical bias.” While the 

broader evidence regarding the ability of researchers to ameliorate hypothetical biases is mixed, 

there is no evidence that such biases are unavoidable, and Murphy et al. (2010) state that there is 

a “small but growing literature is suggesting the absence of hypothetical bias in aggregate in 

induced value settings”.

 A fifth hypothesis is that the water quality levels and changes described in some stated preference 

studies may translate imperfectly to the 100-point water quality index, resulting in measurement 

error in the meta-regression. A related hypothesis is that the water quality changes in the metadata could be correlated with some other omitted variable that also influences marginal WTP.

 A final hypothesis is that respondents’ utility from water quality improvements may 

endogenously depend on water quality in the area in which they live. This could occur in at least 

two ways. First, in a spatial hedonic equilibrium, people are likely to move to areas that where 

the marginal cost (broadly interpreted) of accessing natural amenities is approximately equal to 

the marginal utility benefits provided by those amenities. Alternatively, even if water quality has 

little effect on residential sorting, people and communities may make recreational investments

(e.g., in skills, equipment, or facilities) that are appropriate for their local environmental 

conditions. For example, towns located in areas with very poor water quality might not build 

public swimming beaches—thus reducing the marginal benefit of large improvements in water 

quality.

Overall, the data from the meta-analysis do not provide any simple way of testing or ruling out these 

different possibilities. This makes choosing a single value for the water quality change parameter in 

MRM2-S difficult. Our recommendation is instead to conduct a sensitivity analysis, using the range of 

water quality change scenarios found in the meta-dataset. The purpose of this sensitivity analysis would 

be to explore the range of values that would be generated if EPA were to conduct a well-designed stated 

preference survey designed to elicit marginal WTP for the types of water quality improvements expected 

under future CWA regulations.

To aid in choosing appropriate water quality change values, Figure 3, on the following page, presents a 

histogram of the water quality changes valued in studies in the meta-dataset. The figure shows that the 

water quality changes in the dataset range from +2.5 units to +57 units. The average change in the dataset

is +20.7 units. To put these values in context, note that the water quality index ranges from a minimum of 
12

0 (for water that is so polluted that it has no uses) to 100 (for water that is drinkable without treatment). 

For example, an index value of 45 means that rough fishing is possible, and an index value of 50 means 

that game fishing is possible. Thus, on this scale, even relatively small changes can provide noticeable 

improvements. 

Based on the range of water quality changes in the meta-data, we believe that a reasonable sensitivity 

analysis might use MRM2-S to predict marginal WTP under the following three alternative values of the 

water quality change parameter: +5 units, +20 units, and +50 units.

Figure 2. Distribution of Water Quality Change in the Meta-Dataset

3.1.5 Estimated Marginal Willingness-To-Pay

To illustrate the implications of different choices of the change in water quality parameter, this section

presents estimates of marginal WTP based on MRM2 and MRM2-S.

Figure 3 shows predicted marginal WTP from MRM2-S, based on the specification that is linear in water 

quality change. The figure presents sensitivity analysis results for three different values of the water 

quality change methodology variable: +5 units, +20 units, and +50 units. For comparison, the figure also 

present results from MRM2 (which does not depend on water quality change). Overall, the figure shows 

that when MRM2-S is evaluated at a water quality change of +20 units, it produces results that are very 

similar to MRM2. This is not surprising, given that MRM2 is essentially evaluating marginal WTP at the 

average water quality change in the dataset (which is +20.7 units). The main difference is that under 

MRM2-S, marginal WTP has a flatter slope with respect to water quality. 

Figure 3 also shows that when MRM2-S is evaluated using a water quality change parameter of +50 units, 

it produces marginal WTP values that are roughly 50% lower than when MRM2-S is evaluated at +20 
13

units. Conversely, when MRM2-S is evaluated at +5 units, it produces marginal WTP values that are 

roughly 40% to 50% higher than when MRM2-S is evaluated at +20 units. 

Figure 4 presents a similar comparison of MRM2 and MRM2-S, but using the version of MRM2-S based 

on the specification that includes the log of water quality change. Overall, one main difference is that this 

version of MRM2-S produces marginal WTP curves that are even flatter with respect to water quality. At 

the same time, marginal WTP values from this model are more sensitive to the choice of the water quality 

change parameter. In particular, marginal WTP values are substantially higher when the water quality 

change parameter is set equal to +5 units.

Figure 3. Predicted Marginal WTP, for MRM2-S with Water Quality Change
14

Figure 4. Predicted Marginal WTP, for MRM2-S with Log Water Quality Change

The marginal WTP values predicted here for MRM2 are in the range of $3 to $11 per unit. This range is 

fairly comparable to the average WTP per water quality unit from the meta-dataset, which is $6.30. 

However, this range is quite a bit higher than the predicted MRM2 values from the peer review report, 

which ranged from approximately $1 to $3 (see Fig 11 from that report). These differences are due to 

changes in the values assigned to the control variable parameters in MRM2, and to the fact that the model 

is relatively sensitive to the choices of certain variables.

As a reference, Table 2 shows the parameter values at which the meta-regression model was evaluated, 

for the purposes of generating the marginal WTP values shown in Figures 3 and 4. Other choices for 

these parameters are possible, and we welcome EPA’s guidance on selecting appropriate values.
15

Table 2. Assigned Values for Meta-Regression Parameters

Parameter Assigned Value

Ce 0.518

Thesis 0

Lnyear 2.525

Volunt 0

outlier_bids 1

Nonparam 0

non_reviewed 0

lump_sum 0

WTP_median 0

Northeast 0.101

Central 0.389

South 0.327

Nonusers 0

lnincome1 10.894

mult_bod 0

river1 1

swim_use 0.264

Gamefish 0.057

boat_use 0.114

ln_ar_agr -1.267

ln_ar_ratio1 1.238

sub_frac 0.0239

Qavg2 varies

quality_ch varies

lnquality_ch varies

_cons 1

3.1.6 Application to the Steam Electric Rule

This section uses the MRM2-S marginal WTP values from the previous section to generate back-of-theenvelope estimates of the yearly benefits of the Steam Electric Rule, once it fully takes effect. Table 3

presents the results of these calculations, as well as estimates based on MRM2. Note that this table makes 

a number of simplifications. In particular, it assumes that all 85 million households affected by the rule 

reside in areas in which baseline water quality is 50 on the water quality index. It also assumes that the 

water quality improvement is a homogenous 0.1 units everywhere. These values are approximations, and 

should be interpreted only as order-of-magnitude estimates of the benefits of the Steam Electric 

Regulation. They do not account for the time required for roll-out of the regulation, or for discounting.

Table 3 shows that under MRM2, annual benefits are predicted to be $42.5 million per year. When 

MRM2-S is evaluated at a water quality change parameter of +20 units, it generates similar predictions: 

$46.4 million per year for the linear water quality version, and $41.1 for the log water quality version. 

Under the linear water quality version of MRM2-S, benefits range from $20.3 million per year to $69.9 

million per year, when evaluated using water quality change parameter values of +50 units and +5 units, 

respectively. Under the log water quality version of MRM2-S, benefits are higher, and range from $22.4 

million per year to $103.3 million per year.
16

Table 3. Approximate Benefits Estimates for the Steam Electric Rule

(a) (b) (c) (d) (e)

Affected 

Households

Baseline 

Water Quality 

Index

Average 

Change in 

Water Quality 

Index

Marginal WTP 

per Year, at 

Baseline 

Water Quality

Yearly 

Benefits 

(millions)

MRM2 85 million 50 0.1 $5.00 $42.5

MRM2-S with ΔWQ

 ΔWQ=+5 85 million 50 0.1 $8.22 $69.9

 ΔWQ=+20 85 million 50 0.1 $5.46 $46.4

 ΔWQ=+50 85 million 50 0.1 $2.39 $20.3

MRM2-S with ln(ΔWQ)

 ΔWQ=+5 85 million 50 0.1 $12.15 $103.3

 ΔWQ=+20 85 million 50 0.1 $4.84 $41.1

 ΔWQ=+50 85 million 50 0.1 $2.63 $22.4

Notes: Note that this table presents estimates of yearly benefits once the Steam Electric Regulations reaches full 

effect. Due to discounting and the time required for the roll-out, actual annualized benefits will be considerably 

smaller (i.e., by perhaps 30%).
17

4. Next Steps

This memo presents our recommendations for incorporating a water quality change variable into the 

MRM2 meta-regression model, and treating it as a methodological characteristic. We believe that there 

are good theoretical and empirical reasons for doing so. The resulting MRM2-S model satisfies the 

adding-up property, while at the same time controlling for how survey respondents’ implied marginal 

WTP values are affected by the scope of the resource change presented to them in the survey. At the 

same time, the MRM2-S model provides a substantial increase in R2relative to MRM2. It also provides 

EPA with a way to conduct a sensitivity analysis that accounts for the full range of marginal WTP values 

that might be generated using reasonable stated preference techniques.

We would be happy to discuss the material in this memo further, or to run alternative versions of the 

analysis based on other sets of input parameters. We look forward to EPA’s guidance on whether and 

how to implement and apply this version of the meta-regression model.
18

References

Champ, P.A., and R.C. Bishop. 2001. “Donation payment mechanisms and contingent valuation: An 

empirical study of hypothetical bias.” Environmental and Resource Economics 19:383–402.

Champ, P.A., R. Moore, R.C. Bishop. 2004. “Hypothetical Bias: The Mitigating Effects of Certainty 

Questions and Cheap Talk.” Selected Paper prepared for presentation at the American Agricultural 

Economics Association Annual Meeting, Denver, Colorado.

Cummings, R. and L. Taylor. 1999. Unbiased Value Estimates for Environmental Goods: A Cheap Talk 

Design for the Contingent Valuation Method. American Economic Review 89:649-665.

Diamond, Peter A., Jerry A. Hausman, Gregory K. Leonard, and Mike A. Denning. 1993. “Does 

Contingent Valuation Measure Preferences? Experimental Evidence.” In Contingent Valuation, A Critical 

Assessment, ed. J.A. Hausman, 41–89. Amsterdam: Elsevier.

Johnston, R.J. 2006. Is Hypothetical Bias Universal? Validating Contingent Valuation Responses Using a 

Binding Public Referendum. Journal of Environmental Economics and Management 52(1): 469-481.

Murphy, J.J., P.G. Allen, T.H. Stevens, and D. Weatherhead. 2005. A Meta-Analysis of Hypothetical Bias 

in Stated Preference Valuation. Environmental and Resource Economics 30(3): 313–25.

Ready, R.C., P.A. Champ and J.L. Lawton. 2010. Using Respondent Uncertainty to Mitigate Hypothetical 

Bias in a Stated Choice Experiment. Land Economics 86(2): 363-381.