Non-Market Values for Alternative 

Operations of the Glen Canyon Dam:

Explorations in Choice and Valuation

Phase II Project Research and Findings

July 2014

Hank	C.	Jenkins-Smith,	Carol	L.	Silva,	Deven	Carlson,

Joseph	Ripberger,	Kuhika	Gupta,	and	Nina	Carlson

Center for Energy, Security and Society

University of Oklahoma
Contents

1. Introduction	and	Overview	of	the	Study..................................................................4

1.1 Study	Limitations ...............................................................................................................6

1.2 Outline	of	Report................................................................................................................7

2. Background—The	Research	Puzzle ....................................................................... 10

3. Research	Design	and	Experiments ......................................................................... 13

4. Replication	Treatment................................................................................................ 18

5. Baseline	Randomized	Information	Treatment................................................... 23

6. Referendum	Options	Treatment............................................................................. 26

7.	 Additional	Value	Consideration	Treatments ..................................................... 33

7.1	 Native	American	Tribes ............................................................................................... 35

7.2	 Rural	Western	Communities ...................................................................................... 38

7.3	 Native	American	Tribes	+	Rural	Western	Communities................................... 40

8.	 Hypothetical	Air	Emissions	Treatment................................................................ 42

9.	 Validity	Assessments	of	Study	Results ................................................................. 47

10.	 Summary	and	Conclusions..................................................................................... 54

Appendix	A:	Survey	Data	Collection............................................................................... 57

Appendix	B:	Frequency	Report	and	Data	Summary ................................................. 63

Appendix	C:	Experimental	Survey	Treatments .......................................................... 68

References .............................................................................................................................. 79
Tables and Figures

Tables

Table	1:	Experimental	Treatments ................................................................................................. 15

Table	2:	Design	Features	of	Experimental	Treatments.......................................................... 16

Table	3:	Replication	Treatment	Results—No	Cost................................................................... 21

Table	4:	Replication	Treatment	Results—Cost	of	$25............................................................ 22

Table	5:	Randomized	Information	Treatment	Results........................................................... 25

Table	6:	Referendum	Options	Treatment	Results .................................................................... 32

Table	7:	Native	American	Treatment	Results............................................................................. 37

Table	8:	Rural	Western	Community	Treatment	Results........................................................ 40

Table	9:	Native	American	+	Rural	Western	Community	Treatment	Results ................ 41

Table	10:	Hypothetical	Air	Emissions	Treatment	Results .................................................... 46

Table	11:	Sources	of	Operational	Preferences	and	WTP....................................................... 49

Table	12: Analysis	of	Attention	and	Effort................................................................................... 52

Table	A	1:	Demographic	Representativeness	of	Respondents	(Weighted)................... 61

Figures

Figure	1:	Single	Proposal	Format	for	Valuation ........................................................................ 26

Figure	2:	Two-Option	Format	for	Valuation ............................................................................... 29
INTRODUCTION AND OVERVIEW OF THE STUDY PAGE | 4

1. Introduction and Overview of the Study

Understanding	 non-market	 values	 affected	 by proposed	 operational	 changes	 for the	 Glen	

Canyon	 Dam	 (GCD)	 has	 been	 the	 topic	 of	 considerable	 prior	 investigation,	 for	 both	

recreational	use	and	environmental	non-use	values	(e.g.,	Bishop	et	al.,	1987;	and	Welsh	et	

al.,	 1995).	 In	 addition	 to their	 role	 as investigations	 of	 market	 impacts	 (Douglass	 &

Harpman,	 1995),	 these studies were	 completed	 as	 part	 of	 prior	 federally-mandated	

environmental	 assessments on	 GCD	 operations	 (Lovich	 & Melis,	 2007).	 They	 were	

important	in	bringing	non-market	values	into	the	environmental	and	economic	analysis	of	

managing	 large	 coupled	 human	 and	 natural	 systems,	 such	 as	 the	 Colorado	 River	 Basin	

(Harpman	et	al.,	1995;	Loomis	et	al.,	2005).	But	as	with	other	components	of	evaluating this	

complex	system	(e.g.,	ecological	effects),	these	a	priori studies	leave	many	unresolved	issues,	

and	merit	re-visiting	(see	Lovich	& Melis,	2007).		

In	 2014,	 as	 part	 of	 required	 updates	 of	 environmental	 assessments,	 the	 evaluation	 of	

potential	changes	in	the	operation	of	the	GCD	has	raised	important	theoretical	and	practical	

questions	 about	 the	 range	 of	 societal	 values	 that	 may	 be	 affected,	 and	 how	 they	 can	 be	

measured	 for	purposes	of	informing	public	policy.	Such	questions	can	be	seen	as	part	of	a	

longer	 trend	in	 expanding	 the	 perspective	 of	 socio-economic	 analyses	in	 water	 resources	

management	and	policy	(e.g.,	Castle,	2000).	This	report	describes	a	pilot	study	designed	to	

test	for	the	sensitivity	of	members	of	the	U.S. public	to	alternative	non-market	and	non-use	

values	 that	 may	 be	 affected	 by	 dam	 operations.	 The	 study	 also	 develops	 and	 evaluates	 a	

methodology	 for	 future	 estimations	 of	 public	 willingness	 to	 pay	 for	 alternative	 dam	

operations	that	makes	possible	the	incorporation	of a	wider	array	of	value	dimensions	than	

have	been	considered	in	prior	studies.

Changing	the	way	the	GCD	is	operated,	in	the	interest	of	“repurposing”	the	way	the	Colorado	

River	is	used,	will	have	a	range	of	potential	effects	that	may	have	diverse	value	implications	

for	U.S. residents.	Some	of	 these	value	changes	can	be	observed	in	market	prices	(e.g.,	 the	

change	in	value	of	 the	electricity	produced	at	 the	GCD)	and	others	in	changes	in	 the	value	

and	 quantity	 of	 non-priced	goods	 that	 consumers	 of	 resources	 use	 (such	as	 recreation	 on	
INTRODUCTION AND OVERVIEW OF THE STUDY PAGE | 5

the	Colorado	River	or	Lake	Powell).	Our	interest	is	chiefly	in	 those	values	 that	are	neither	

included	in	market	prices	paid	for	goods	and	services	nor related	directly	to	an	individual’s

use	of	the	resource	– termed	non-market, non-use	values	(Loomis, 2014).	Our	primary	focus	

is	on	how	residents	of	 the	U.S. will	evaluate	 the	potential	bundles of	changes	in	conditions	

affecting	 non-market,	 non-use	 values	 that	 are	 affected when	 operational	 decisions	 may	

affect	multiple	kinds	(or	dimensions)	of	value.	The	conceptual	and	methodological	challenge	

is	 that	 changes affecting	 multiple	 dimensions	 of	 value	 may,	 from	 the	 perspective	 of	

residents	of	the	U.S.,	result	in	expressed	values	that	range	from	negative	to	positive.	A	valid	

approach	 for	 measurement	 of	 non-use	 values	 for	 changing	 operations	 of	 the	 GCD	 must	

therefore	 evaluate	 relevant	 dimensions	 of	 value	 and	 provide	 for	 a	 range	 of	 values	 – from	

negative	to	positive	– associated	with	those	changes.

Using	 the	 survey-based	 contingent	 valuation	 (CV)	method,	 the	 pilot	 study	 described	 here	

was	 designed	 to	 ascertain	 whether	 a	 cross-section	 of	 residents	 in	 the	 U.S. hold	 non-use	

values	for	a	range	of	possible	changes	that	could	occur	as	a	result	of	operational	changes	in	

hydropower	production	at	the	GCD.	Prior	CV	research	(Welsh et	al., 1995)	has	focused	on	a	

subset	 of	 non-use	 values	 for	 the	 preservation	 of	 river-related	 resources	 – including	

riverside	beaches,	downstream	habitat,	and	cultural	sites	– that	might	be	affected	by	dam	

operations.1 This	study	broadens	the	assessment	of	potential	dimensions	of	non-use	values	

in	 a	 number	 of	ways.	 The	 first	is	 to	 address the potential	 for	 non-use	 values	 held	 by	U.S.

residents	related	 to	prospective	changes	affecting	rural	communities	 that	have	adapted	 to	

the	 resources	 and	 economic	 opportunities	 afforded	 by	 the	 operation	 of	 the	 dam.	 The	

potential	 non-use	 values	 of	 interest	 in	 this	 case	 are	 not	 jobs	 or	 economic	 activity	 per	 se;	

rather	we	are	considering	the	value	placed	on	changes	in	the	sustenance	for	what	the	public	

may	perceive	to	be	important	or	unique	rural	cultures	and	ways	of	life (e.g.,	see	Berrens	et	

al.,	 1998). This	 CV	 study	 also	 evaluates	 non-use	 value	 concerning	 a	 purely	 hypothetical	

question:	 if	 the	 change	 in	 dam	 operations	 were	 to	 increase	 reliance	 on	 electricity	 from	

power	 plants	 utilizing fossil	 fuels,	 what	 non-use	 value	 might	 U.S. residents	 place	 on	

avoidance	 of	 that	 increase?	 For	 both	 of	 these	 lines	 of	 inquiry,	 we	 have	 built	 upon	 the	

theoretical	 framework	 for	 non-use	 value	 that	may	 be	 affected	 by	 changes	in	 hydropower	

operations	as	developed	by	Loomis	(2014).

																																																							 1 Focusing	on	the	subset	of	value	dimensions,	Welsh	et	al.	(1995:	5-17)	concluded	that	U.S.	residents	

would	be	willing	to	pay	over	$2.2	billion	annually	to	modify	GCD	operations	to	achieve	a	“moderate	

fluctuating	flow”	regime	(also	see	Lovich	&	Melis,	2007).
INTRODUCTION AND OVERVIEW OF THE STUDY PAGE | 6

Our	 focus	 on	 the	 potential	 for	 complex	 changes	 associated	 with modifying hydropower	

operations	required	that	we	reconsider	how	to	measure	non-use	values	when	the	alteration

results	 in	 a	 bundle of	 changes	 across	 multiple	 non-use	 attributes.	 When	 evaluating	 these	

bundles	 of	 non-use	attributes,	individuals	may	express	 positive	 or	 negative	 values	 for	 the	

change.	A	common	approach	used	for	measuring	non-market	values,	contingent	valuation	of	

willingness	to	pay	(WTP) responses,	as	often	applied,	does	not	readily	incorporate	negative	

values	 placed	 on	 what	 may	 be	 a	 non-rejectable	 “good”;	 however,	 a	 limited	 number	 of	

studies	 have	 attempted	 to	 address	 this	 concern (see,	 e.g., Bohara,	 Kerkvliet, and	 Berrens,	

2001;	Atkinson	et	al.,	2004;	Loureiro	et	al.,	2004).	 Indeed,	 those	who	would	not	choose	 to	

support	 the	 good	 even	 if	 it	 costs them	 nothing	 are	 typically	 assumed	 to	 place	 no	 value	

(either	 positive	 or	 negative)	 on	 the	 good.	 This	 may	 be	 a	 reasonable	 assumption	 if	 the	

proposed	change	can	be	rejected	or	easily	avoided	by	 the	individual	 (e.g.,	don’t	buy,	don’t	

use,	don’t	participate,	choose	a	readily	available	alternative,	etc.).	However,	in	the	case	of	a	

non-rejectable	 good	 (Loureiro	 et	 al.,	 2004),	 such	 as	 in the	 case	 of	 a	 change	 in	 dam	

operations	 involving	 a	 complex	 bundle	 of	 attributes,	 this	 may not	 be	 a	 reasonable	

assumption.	 For	 that	 reason,	 this	 CV	 study	 required	 the	 development	 of	 a	 valuation	

approach	that	permits	individuals	to	choose	among	options	(e.g.,	change	or	no	change),	then	

provide	a	valuation	response to	that	option,	rather	than	valuing	only	one	proposed	course	

of	action.	This	type of	approach	permits more	 flexible	and	comprehensive	assessments of	

preferences	 and	WTP	 for	 either	 changing	 dam	 operations	 or	 for	 keeping	 dam	 operations	

unchanged.

1.1 Study Limitations

As	is	 described	in	 detail	in	 the	 next	 section,	 this	 pilot	 study	 focused	 on	 the	 differences	in	

survey	responses	to	a	specific	set	of	experimental	treatments.	However,	the	study	was	not	

intended	to	provide	a	means	 for	estimating	WTP	or	 for	making	inference	to	the	expressed	

values	of	U.S. population.	

In	order	 to	 focus	on	multiple	experimental	 treatments	encompassing	variation	in	both	 the	

method	of	eliciting	WTP	and	the	inclusion	of	diverse	value	dimensions,	we	limited	the	range	

of	expressed	preferences	to	(a)	preferred	GCD	operations	at	no	cost	and	(b)	preferences	if	

the	 cost	 to	 the	 respondent	was	 $25	 per	 year for	 the	 foreseeable	 future in	added	 taxes.	 In	
INTRODUCTION AND OVERVIEW OF THE STUDY PAGE | 7

this	way,	we	were	able	to	collect	sufficient	numbers	of	responses	for	each	treatment	to	test	

for	the	magnitude	and	statistical	significance	of	differences	in	expression	of	value	across	the	

different	experimental	treatments.	However,	because	we	were	unable	to	include	a	sufficient

range	 of	 dollar	 values	 representing	 costs	 to	 respondents	for each	 option,	 our	 data	 do	 not	

permit	us	to	estimate	WTP	under	each	of	the	experimental	conditions.	

In	 addition,	 the	 data	 were	 collected	 from	 a nationwide, online	 panel	 of willing	 Internet	

survey	respondents	– both	to	facilitate	the	experimental	treatments	and	to	reduce	the	cost	

and	time	required	for	data	collection.	Though	the	data	closely	match	the	national	Census	on	

an	array	of	key	demographic	attributes,	these	data	cannot	be	considered	a	random	sample	

of	 the	U.S.	 public,	 and	 therefore	 inference	 from	 our	 sample	 results	 to	 the	U.S.	 population	

must	 be	 made	 with	 caution.	 See	 Section	 2	 and	 Appendix	 A	 for	 further	 discussion	 of	 the	

sample,	 its	 representativeness	 of	 the	 U.S. public,	 and	 the	 weighting	 process	 used	 for	 the	

analyses	in	subsequent	sections	of	this	report.

In	sum,	the	pilot	study	described	in	 this	report	is	intended	 to	provide	guidance	 for	 future	

efforts	 that	 would	 evaluate	 a	more	limited	 number	 of	 scenarios	 using	 a	methodologically	

appropriate	range	of	dollar	costs.	The	results	described	in	this	report	should	be	understood	

in	that	light.

1.2 Outline of Report

This	 report	 details	 the	 pilot	 study	 and	 findings.	 Sections	 2 through	 6 are	 primarily	

methodological	 in	 nature.	 Section	 2	 describes	 the	 theoretical	 and	 practical	 puzzle	 that	

underlies	this	study	– how	do	people	formulate	preferences	for	policies	involving	change	in	

complex	coupled	human	and	natural	systems?	How	is	it	that,	when	hydropower	is	generally	

seen	as	a	benign	and	valuable	resource,	other	studies (e.g.,	Welsh	et	al., 1995)	have	 found	

that	the	public	is	willing	to	pay	billions	of	dollars	to	constrain	the	use	and	lower	the	value	of	

hydropower	produced	at	the	GCD?	These	are	the	kinds	of	questions	that	motivate	this	study.	

Section	3	describes	the	research	design,	data	collection	process,	and	the	set	of	experiments	

fielded	 in	 the	 study.	 Section	 4 focuses	 on	 the	 results	 of	 our replication	 of	 earlier	 work	

undertaken	 to	 measure	 WTP	 for	 downstream	 ecological	 and	 cultural	 resources.	 The	

purpose of	 this	 section	is	 to	 evaluate	 data	 that	 were	 collected	 using	 a	 survey	 design	 that	
INTRODUCTION AND OVERVIEW OF THE STUDY PAGE | 8

closely	 mirrors	 work	 done	 in	 prior	 studies.	 If	 the	 results	 of	 the	 replication	 match	 prior	

studies,	 they	 can	 provide	 a	 baseline	 for	 comparison with	 alternative	 approaches	 for	

presenting	 information	 and	 eliciting	 choices	 regarding	 GCD	 operations.	 Section	 5 briefly	

describes	results	 from	a	comparison	survey	design	 that	randomizes	 the	ordering	in	which	

the	 information	 about	 the	 study	 background	 and expected	 effects	 of	 changing	 dam	

operations	is	presented	to	respondents.	Validation	that	randomization	provides	consistent	

results	 is	 necessary	 for	 the	 experiments	 described	 subsequently,	 in	 which	 additional	

dimensions of	potential	changes	are	introduced	to	supplement	 the	ecosystem	and	cultural	

resource	effects	described	in	earlier	studies.	Section	6 describes	an	experiment	in	which	the	

elicitation	of	respondent	preferences	and	WTP	are asked	for	two	discrete	options	– change	

in	 dam	 operations	 or	 continuation	 of	 current	 operations	 – rather	 than	 asking	 only	 for	

valuation	of	a	specified	change.	As	described	in	Section 6,	the	validation	of	this	innovation	is	

necessary	 for	measurement	of	preferences	when	it	is	possible	 that	some	respondents	will	

not	only	prefer	not to	change	dam	operations,	but	may	be	willing	to	pay	to	avoid change.	

Sections	7	and	8 focus	on	the	primary	substantive	experiments	fielded	in	this pilot	study.	In	

Section	7 we	describe	the	results	of	three	experiments	in	which	additional	value	dimensions	

are	 provided	 to	 respondents.	One	experiment	 describes	 potential	effects	 of	 changing	 dam	

operations	on	rural	farming	and	ranching	communities (also see	Jacobs,	2011) and	another	

focuses	on	possible	effects	on	Native	American	 tribes	 that	currently	 receive	benefits	 from	

electricity	produced	at	GCD.	A	third	experiment	combines	the	description	of	effects	on both	

groups.	These	tests	are	intended	to	determine	whether	inclusion	of	these	local	community	

dimensions	 of	 value	will	 substantively	 change	 the	expression	 of	 preferences	and	 value	 by	

the	survey	respondents. Section	8 shifts	the	focus	from the	results	of analysis	of	responses	

to	 a	 purely	 hypothetical	 scenario,	 in	 which	 changes	 in	 dam	 operations	 are	 described	 as	

resulting	in	reductions	in	overall	production	of	hydropower,	with	the	deficit	being	replaced	

with	 power	 from	 conventional	 fossil	 fuel	 plants	 (consideration	 of	 effects	 on	 local	

communities, as	described	in	Section	7, are	omitted	from	this	experiment).	Evidence	of	nonmarket	and	non-use	values	associated	with	such	a	change	would	be	of	relevance	for	cases	in	

which	 changes	 in	 hydropower	 capacity	 are	 being	 considered,	 such	 as	 the	 removal	 or	

addition	of	generators.
INTRODUCTION AND OVERVIEW OF THE STUDY PAGE | 9

Section	9	turns	to	the	question	of	why	the	survey	respondents	chose	to	support	changing	or	

maintaining	the	current	operations of	the	GCD.	Using	a	set	of	models	that	relate	individual	

characteristics	 (income,	 education,	 rural	 residence),	 beliefs	 (environmental	 views,	

perceptions	 of	 hydropower),	 and	 intended	 behaviors	 (expectations	 of	 visiting	 the	 Glen	

Canyon	in	the	future),	along	with	the	variations	in	experimental	treatments,	to	preferences	

for	 GCD	 dam	 operations, we	 show	 that	 differences	 in	 preferences	 are	 systematically	 and	

reasonably	 related	 to	 respondents’ values	 and	 characteristics,	 as	 well	 as	 the	 variation	 in	

experimental	treatments.

Section	10 concludes	with a	summary	of	our	study	 results.	Appendices	A	and	B	provide	a	

description	 of	 the	 data	 collection	 and	 representativeness	 of	 the	 sample,	 and	 the	 survey	

wording	 and	 response	 frequencies,	 respectively. Appendix	 C	 shows	 the	 specific	 wording	

and	formatting	of	the	experimental	treatments	in	visual	form,	as	fielded	for	this	study.
BACKGROUND—THE RESEARCH PUZZLE PAGE | 10

2. Background—The Research Puzzle

For members	 of	 the	U.S.	public, hydropower	is	broadly	 understood	 to	 be a	 stable,	 secure,	

and	renewable	form	of	energy.	A	2012	nationwide	survey	found	that,	when	apprised	of	the	

distribution	 of	 current	 energy	 sources,	 U.S. residents	 would	 prefer	 to	 see	 reliance	 on	

hydropower	 rise	 from	 3%	 to	 20%	 of	 the	 overall	mix	 of	 energy	 sources	 (Herron,	 JenkinsSmith, & Silva,	2012).	Why	would	Americans	prefer	to	see	such	an	increase?	In	large	part	it	

appears	 to	 be	 because	 of	 the	 perceived	 attributes	 of	 hydropower.	 In nationwide	 surveys	

taken	 in 2008 and	 2014,	 large	 respondent	 majorities	 consistently	 characterized	

hydropower	 as	 clean,	 safe,	 and	 renewable.	 Put	 simply,	 Americans	 appear	 to	 prefer	

hydropower	because	it	– like	solar	and	wind	power	– is viewed	as	beneficial	to	society	and	

the	environment	(Jenkins-Smith	&	Herron, 2008).	

Considering	 the	 positive	 views	 of	 hydropower	 held	 by	 a	large	majority	 of	the	 public,	it	is	

notable that	studies	of	social	preferences	seem	to	indicate	a	substantial	willingness	to	pay

(WTP) to	constrain	 the	production	of	hydropower	at	 the	GCD (Welsh	et	al., 1995). Why	is	

that?	In	WTP	studies,	public valuation	 for	hydropower	is	likely	 to	be	highly	 responsive	 to	

the	 value	 dimensions	 of	which	the	members	 of	 the	 public are	 apprised,	 and	 the	 resulting	

tradeoffs	among	values	that	are	implied.	If	so,	the decision	to	include or	exclude potentially	

relevant	value	dimensions from	the	choice	scenario is	of	central	importance	in	interpreting	

the	study	results.	In	 this	research	we	systematically	analyze	several	 factors	 that	may	help	

explain	the	finding	that,	in	spite	of the	strikingly	positive	public perceptions	of	hydropower,

the	 public	 would	 be	 willing	 to	 pay	 to	 constrain	 production	 of	 hydropower	 at	 the	 GCD.	

Specifically,	we	explore	how	the	presentation	of	information,	the	approach	used	to	structure	

choices,	and	the	omission	of	potentially	relevant	dimensions of	value in	previous	work may	

have	contributed to	the	puzzle	described	above. But, some	background	context	is	necessary

to	explain	why	the	kinds	of	information	and	structure	of	choices	would	matter	for	how	the	

public	can	express	non-use	values	 for	complex	policy	decisions	of	the	kind	represented	by	

the	operation	of	the	GCD.

The	GCD represents	an	important	component	of	a	much	larger	coupled	human	and	natural	

system	 on	 the	 Colorado	 River	 (Jacobs,	 2011). Completed	 in	 1966,	 the	 GCD	 provides	 a	
BACKGROUND—THE RESEARCH PUZZLE PAGE | 11

renewable	 supply	 of	 (non-fossil	 fuel based)	 hydroelectricity,	and	 regulates	 flows	 between	

the	 upper	 and	 lower	 basins	 of	 the	 Colorado	 River.	 Over	 time,	 local	 communities	 and	

ecosystems	 interact	 with,	 and	 are	 affected	 by,	 the	 regular	 pattern	 of	 operations	 in	 these	

large	 coupled	 systems.	 	 These	 become	 the	 working	 landscapes	 of	 the	 river,	 and	 an	

increasingly	important	part	of	conservation	debates	(Kareiva,	Marvier,	&	Lalaz, 2012).

As	institutional	arrangements	and	specific	operating	patterns	for	 the	dam	are	periodically	

reviewed	 and	 re-evaluated,	 an	 important	 management	 consideration	 or	 step	 was	 the	

attempt to	 account	 for	 the	 non-market	 environmental	 values	 that	 might	 be	 affected,	

including	both	use	values	and	non-use	values	(Loomis	et	al.,	2005).		For	example,	prominent	

CV	 studies	 on	 the	 GCD	 in	 the	 1990’s	 estimated	 non-market	 monetary	 values	 for	 posited	

changes	in	both	downstream	and	reservoir	recreational	uses	(Bishop	et	al.,	1987).	Further,	

as	a	component	of	environmental	assessments	in	 the	1990’s,	Welsh	et	al.	(1995)	provided	

an	important	 early	CV	 study	 on	 non-market,	 non-use	 values	 associated	 with	 downstream	

environmental	and	ecosystem	effects from	proposed	changes	on	GCD	operations.		

In	 the	 intervening	 decades,	 our	 understanding	 of	 the	 complex	 regional	 linkages	 in	 large	

coupled	human	and	natural	systems,	as	well	as	connections	to	broader	global	impacts	from	

our	choices	about	the	energy	production	mix,	has	only	increased	(Liu	et	al.,	2007).		To	wit,	

in	roughly	the	last decade	the	term	“Anthropocene”	has	emerged	as	a	buzzword	in	geologic	

chronology	 (as	 distinguished	 from	 the	 Holocene)	 to	 better	 recognize	 our	 current	 human	

impacts	 and	 interactions	 with	 earth	 systems (Crutzen,	 2002).	 As	 scientists debate	 the	

merits of	epochal	 change,	it	is	 clear	 that	 human	welfare	is	increasingly	 connected	 to	 both	

regional	and	global	change.	Choices	about,	for	example,	old	growth	forest	protection	in	one	

region might	 affect	 the	 market	 for	 hardwoods	 on	 another	 continent	 (e.g., Chimeli	 et	 al.,	

2012),	 or	 choices	 about	 our	 regional	 energy	 mix	 might	 affect	 global	 climate	 change,	 or	

operational	changes	in	one	section	of	a	large	regulated	river	system	may	affect	communities	

and	 working	 landscapes	 both	 up- and	 downstream. As	 such,	 this	 connects	 to	 broader	

debates	 about	 what	 conservation	 in	 the	 Anthropocene	 actually	 means	 (Gillings	& HaganLawson,	 2014;	Kareiva	 et	 al.,	 2012).	Or	in	 our	 context,	it	 connects	 to	 questions	 about	 the	

multiple	 dimensions	 that	 attempts	 at	 “valuation	 in	 the	 Anthropocene”	 might	 need	 to	

consider.
BACKGROUND—THE RESEARCH PUZZLE PAGE | 12

We	 are not	 arguing that	 every	 standard	 valuation	 exercise	 needs	 to consider	 broader	

regional	or	global	connections.	But operational	changes	in	large	coupled	human	and	natural	

systems,	which	involve	much	more	than	a	select	set	or	 type	of	sites,	are	likely	 to confront	

such	 questions.	 Further,	 such	 questions	 are	 consistent with	 an adaptive	 management	

perspective,	as	previously	adopted	for	the	GCD	operations	(Lovich	&	Melis,	2007).

Understanding	large	coupled	human	and	natural	systems	requires understanding the	ways	

that	communities	and	working	landscapes	(Kareiva	et	al.,	2012)	become	inextricably	linked	

with	particular	operational	patterns.	Thus,	any	significant	proposed	change	in	operational	

patterns	must	both	expect	and	account	for	the	social	disruption	likely	to	be	brought	about	

with	the	 operational	 change.	 As	 noted	 by	 Bromley	 (2000),	with	 respect	 to	 evolving	 river	

system	management	in	the	U.S.:

“Is	 this	 transition	 in	 water	 policy	 fair	 to	 those	 whose	 lives	 and	 livelihood	 are	

inextricably	 bound	 up	with	 the	 shifting	 purpose	 of	 the	 rivers?	There	is	 no	easy	

answer	 to	 that.	 A	 civilized	 nation	 cushions	 the	 inevitable	 transitions	 for	 those	

caught	in	the	vise	of	shifting	priorities	and	purposes.”	

Communities	have	adapted	to	(optimized	to)	the	prior	human	interventions.	The	question	is	

whether	 the	 public	 at	 large	 holds	 any	 positive	 or	 negative	 non-market	 value	 for	 these	

arrangements,	in	the	form	of	paternalistic	altruism	(Loomis,	2014).

In	 addition,	 the	 possibility	 exists	 that	 other	 policy	 decisions (unlike	 the	 GCD	 case)	 may	

result	in	changes	in	capacity	at	existing	dams,	and	such	change	in	 turn	might	result	in	net	

changes	in	air	emissions	as	hydropower	is	replaced	with	electrical	generation	using	 fossil	

fuels.	Are	there	non-market	values	attached	to	the	implications	of	emissions	resulting	from	

such	a	hypothetical	change?

Below,	we	lay	out	our	strategy	for	investigating	these	exploratory	questions.
RESEARCH DESIGN AND EXPERIMENTS PAGE | 13

3. Research Design and Experiments

We	undertook this	pilot	study	using	data	 from	a	national	Internet-based	survey fielded	by	

the	University	of	Oklahoma’s	Center	for	Energy,	Security	& Society	(hereafter	referred	to	as	

the	Center),

2 in	which	survey	respondents	were randomly	assigned to	receive	one	of	seven	

experimental	 treatments.3 The	 survey	 data	 were	 obtained	 from	 2,465	 respondents who	

were	 recruited	 by	 Survey	 Sampling International (Fairfield,	 CT)	 to	 take	 the	 survey	 in	 an	

online	 format.	The	respondent	recruitment	was	designed	 to	achieve	approximate	national	

representativeness	on	key	demographic	attributes	such	as	gender,	region	of	residence,	and	

the	respondents’	race/ethnicity. For	purposes	of	analysis,	the	data	were	weighted	to	match	

the	 2013	 U.S. Census	 estimates. The	 survey	 instrument	 was	 designed	 and	 tested	 by	 the	

Center,	using	as	a	replication	base	(see	Section	4)	the	information	and	elicitation	questions	

employed	in	 the	survey	 reported	in	Welsh	et	al. (1995).	The	instrument	includes	sections	

that	 ascertain	locational	 and	 demographic	 attributes;	 provide	 background	information	 on	

the	 Glen	 Canyon	 Dam	 and	 the	 potential	 effects	 of	 changes	 in	 GCD	 operations;	 elicit	

preferences	 for	 dam	 operations	 and	willingness	 to	 pay	 $25	in	 additional	 annual	 taxes	 for	

their	 preferred	 option;	 and	 measure	 an	 array	 of	 attitudinal	 attributes	 and	 behaviors	

relevant	 to	valuation	choices.	A	more	detailed	description	of	 the	data	collection	method	is	

provided	 in	 Appendix	 A,	 and	 a	 listing	 of	 question	 wording	 and	 response	 frequencies	 for	

questions	relevant	to	this	report	are	provided	in Appendix	B.

The	 heart	 of	 this	 study	 derives	 from	 the	 comparison	 of	 responses	 to	 the	 seven	 randomly	

assigned	experimental	treatments.	The	first	of	these	treatments	is	designed	to	replicate	the

original	Welsh	 et	 al.	 (1995)	 study	 that	 found	 a	 substantial	WTP	for	 proposed	 changes	 in

operations	at	the	GCD.	This	replication	serves	as	the	baseline	against	which we	compare	the	

results	 of our	 six additional experimental	 treatments. Treatment	 #2 randomly	 varies	 the	

																																																							 2 The	survey	design	and	data	collection	were	undertaken	by	the	Center.	The	University	of	Oklahoma’s	

Office	of	Vice	President	for	Research	provided	the	funding	for	the	survey	data	collection.

3 The	 survey	 was	 fielded	 from	 April	 22-24,	 2014	 and	 took	 an	 average	 of	 26	 minutes	 to	 complete.		

Respondents	were	weighted	to	match	2013	U.S.	Census	estimates.
RESEARCH DESIGN AND EXPERIMENTS PAGE | 14

ordering	of information presented	in	the	replication	treatment.

4This	is	necessary	to	set	the	

stage	for	the	inclusion	(in	randomized	order)	of	information	relevant	to	the	additional	value	

dimensions considered	in	subsequent	treatments.	

Treatment	 #3 changes	 the	 structure	 in	 which	 respondents	 were	 asked	 express	 their	

preference	 for	future	dam	operations. In	the	replication	treatment, individuals	were asked	

to	 vote	“yes” or	“no” on	 a	 single	 proposal	 for	 changing	 dam	 operations, whereas	 in	 this	

treatment	 respondents	were asked	 to	 vote	 for	 one	 of	 two	 options	 for	 operating	 the	 dam.	

Such	an	approach	is	more	useful when	asking	respondents	to	trade	off	important	competing	

value	 considerations.	 In	 addition	 to	 being	 more	 conceptually	 appropriate,	 this	 approach	

allowed us	 to	 assess	 the	 extent	 to	 which	 respondents	 may	 hold	 a	 positive	 WTP5 for	

maintaining	 dam	 operations;	 earlier	 studies	 were	 not	 designed	 in	 a	 manner	 that	 could	

capture	such	preferences (or	dimensions	of	valuation).

The	 remaining four	 treatments were designed	 to	 permit	 assessment	 of	 the	 effects	 of	

including	 additional	 value	 dimensions	 in	 the	 consideration	 of	 dam	 operations.	 Three	 of	

these	treatments	provide	information	and	summarize	the	potential	effects	of	changed	dam	

operations	 on: (Treatment	#4)	Native	 American tribes; (#5)	Rural	Western	 communities;

and	 (#6)	 both	 Native	 American	 tribes	 and	 rural	 Western	 communities.	 Treatment	 #7

provided respondents	 with	 purely	 hypothetical	 information	 about potential	 effects	 of	

changed	 dam	 operations	 on	 air	 emissions	 and	 quality. Table	 1	 summarizes	 the	 seven	

experimental	treatments	and	presents	the	sample	size	for	each.6

																																																							 4 In	 survey	 experiments,	 it	 is	 important	 to	 avoid	 unintended	 bias	 due	 to	 the	 ordering	 of	 the	

presentation	of	information	to	respondents.	The	use	of	a	programmed survey	instrument	applied	on	

an	Internet	platform,	as	was	done	for	this	pilot	study,	readily	permits	this	kind	of	randomization.

5 Eliciting	 individual	 preference	 responses	 for	 such	 positive	 values	 for	 maintaining	 current	

operations	could	either	be	framed	as	willingness	to	pay	to	avoid	any	perceived	loss	due	to	changes	in	

current	operations	 (technically,	a	Hicksian	equivalent	welfare change	measure),	or	a	willingness	 to	

accept	 compensation	 for	 perceived	 loss	 due	 to	 proposed	 changes	 in	 operations	 (technically,	 a	

Hicksian	 compensating	welfare	 change	measure).	 The	measures	 differ	in	 assumed	 property	 rights.	

However,	 if	 public	 funds	 are	 necessary	 to	 implement	 any	 option	 (including	 maintaining	 current	

operations),	then	ultimately	a	WTP	perspective	may	be	necessary.	

6 Note	that,	rather	than	vary	each	dimension	of	value	independently	(as	could	be	implemented	in	a	

choice	 experiment),	 we	 evaluated	 specified	 bundles	 of	 attributes in a	 sequential	 set	 of	 plausible	

treatment	options.	We	opted	for	this	approach	because	(a)	we	sought	to	compare	alternative	bundles	

with	the	results	of	a	specific	bundle	of	value	dimensions	represented	in	prior	research (Welsh	et	al.,

1995),	 and	 (b)	 we	 sought	 to	 present	 respondents	 with	 logically	 consistent	 bundles	 of	 changes	 on	

value	dimensions	rather	than	groups	of	attributes	that	were	randomly	aggregated.
RESEARCH DESIGN AND EXPERIMENTS PAGE | 15

Table	1:	Experimental	Treatments

Treatment Concept Sample	Size

1.	Replication Used	Welsh	et	al. (1995)	language	and	

format,	in	Internet	survey	mode 341

2.	Adds	randomized	

presentation	of	

information	to	#1

Replication of	Welsh	et	al.	(1995) with	

the	exception	of	randomizing	the	order	

of	information

345

3.	Adds	referendum	choice	

options	to	#2

Permits	respondent	to	vote	for	one	of	

two	options,	rather	than	one	“proposal” 352

4.	Adds	Native	American	

effects	to #3

Adds	information	and	potential	effects	

of	changed	operations	on	Tribes 343

5.	Adds	rural	Western	

community	effects	to	#3

Adds	information	and	potential	effects	

of	changed	operations	on	rural	

communities

397

6.	Combines	#4	and	#5 Combines	information	and	potential	

effects	on	Tribes	and	rural	communities 345

7.	Adds	hypothetical	air	

emission	effects	to	#3

Includes	hypothetical	effects	of	

increased	air	emissions	(not combined	

with	#4-6)

342

Table	2 summarizes	the	features	of	each	of	the	seven	treatments	to	which	respondents	are	

assigned.	 Specifically,	 it	 indicates	 whether	 each	 treatment	 has	 randomized	 information	

blocks,	a	vote	on	a	proposal	to	change	operations	versus	a	choice	between	two	options	for	

dam	 operations,	 and	 the	 dimensions	 of	 information	 presented	 to	 respondents.
RESEARCH DESIGN AND EXPERIMENTS PAGE | 16

Table	2:	Design	Features	of	Experimental	Treatments

Treatment

Replication 

of 

Information 

in Welsh et 

al. (1995)

Randomized 

Information 

Blocks

Vote on 

Proposal to 

Change 

Operations

Choose 

Between 

Two 

Options for 

Dam 

Operations

Description 

of Effects on 

Native 

American 

Tribes

Description of 

Effects on 

Rural Western 

Communities

Description 

of Effects on 

Air 

Emissions

1 X X

2 X X X

3 X X

4 X X X

5 X X X

6 X X X X

7 X X X

Using	 data	 derived	 from these	 experimental	 treatments, we	 tested the	 following	 six	

hypotheses,	 two	 of	 which	 were	 methodologically	 oriented	 and	 four	 of	 which	 were	

substantive:

Methodological	Hypotheses:

H1:	Randomizing	the	order	in	which	background	information	is	presented	to	respondents	will	

affect	support	for	changing	dam	operations,	as	well	as	respondents’	willingness	to	pay	$25	for	

such	changes,	relative	to	the	baseline	replication	treatment.

H2:	 Allowing	 respondents	 to select	 one	 of	 two	 options	 for	 operating	 the	 dam,	 rather	 than	

requiring	respondents	to	vote	on	a	proposal	for	changing	dam	operations, will	affect	support	

for	changing	dam	operations,	as	well	as	respondents’	willingness	to	pay	$25	for	such	changes,	

relative	to	the	baseline	replication	treatment.

Substantive	Hypotheses:

H3:	 Providing	 respondents	 with	 information	 on the	 potential	 effects	 of	 changed	 dam	

operations	on	Native	American	tribes	will affect	support	for	changing dam	operations,	as	well	

as	 respondents’	 willingness	 to	 pay	 $25	for	 such	 changes,	 relative	 to	 the	 baseline	 replication	

treatment.

H4:	 Providing	 respondents	 with	 information	 on	 the	 potential	 effects	 of	 changed	 dam	

operations	on	rural	Western	communities	will affect	support	for	changing	dam	operations,	as	
RESEARCH DESIGN AND EXPERIMENTS PAGE | 17

well	 as	 respondents’	 willingness	 to	 pay	 $25	 for	 such	 changes,	 relative	 to	 the	 baseline	

replication	treatment.

H5:	 Providing	 respondents	 with	 information	 on	 the	 potential	 effects	 of	 changed	 dam	

operations	on	both	Native	American	tribes	and	rural	Western	communities	will affect	support	

for	changing	dam	operations,	as	well	as	respondents’	willingness	to	pay	$25	for	such	changes,	

relative	to	the	baseline	replication	treatment.

H6:	 Providing	 respondents	 with	 information	 on	 the	 potential	 effects	 of	 changed	 dam	

operations	 on	 air	 emissions	 will affect	 support	 for	 changing	 dam	 operations,	 as	 well	 as	

respondents’	 willingness	 to	 pay	 $25	 for	 such	 changes,	 relative	 to	 the	 baseline	 replication	

treatment.

In	sum,	the	survey	was	designed	to provide	experimental	evidence	about	the	differences	in	

valuation	 responses	 that	 are	 obtained	 when	 survey	 respondents	 are	 provided	 with	

descriptions	of	different	dimensions	of	 value	considerations	 for	changing	dam	operations.	

Recall	that	the data	for	 this	 study	were	 obtained	 from	 respondents to	 a	 voluntary	 online	

survey	 employing	 willing	participants;	in	 an	ideal	 setting,	 of	 course,	the	 data	 would	 have	

been	 obtained	 from	 subjects	 randomly	 drawn	 from	 a	 list	 of	 individuals	 from	 a	 known

population.	 However,	 recent	 developments	 have	 resulted	 in	 declining	 response	 rates	 for	

surveys of	 all	 modes,	 with	 the	 result	 that	 selection	 biases	 (systematically	 different	

propensities	 for	 people of	 different	 kinds to	 respond	 to	 surveys)	 call	 into	 question	 the	

representativeness	 of	 general	 population	 surveys (Anseel	 et	 al., 2010).	 The	 advent	 of	

Internet	surveys	now	provides a	reasonable	alternative	to	other	survey	modes	(see	Berrens	

et	al., 2003,	as	applied	to	CV	studies), and recent	research	has	demonstrated that	carefully	

weighted	 non-probability	 samples	 can	 provide very	 close	 estimates of population	

parameters	(Wang	et	al., 2014). These	developments	underlie	our	use	of	online	survey	data	

for	the	analyses	in	the	following	sections.
REPLICATION TREATMENT PAGE | 18

4. Replication Treatment

To	 the	 extent	 practicable	 given	 the	 difference	in	 survey	modes—the	 original	Welsh	 et	 al.	

(1995)	CV	survey	was	administered	via	mail	whereas	our	survey	was	Internet-based—our	

replication	 treatment	provided	respondents	with	 the same	information	in	 the	same	visual	

format	as did	the	original	Welsh	et	al.	(1995) survey (see	Appendix	C).		Because	of	potential

survey	 mode	 effects, as	 well	 as	 the	 possibility	 of	 preferences	 evolving	 over	 time,	 this	

replication	 treatment	 was	 not	 intended	 to	 precisely	 reproduce	 the	 estimates	 from	 the	

Welsh	 et	 al.	 (1995)	 CV	 study. Instead,	its	 primary	 purpose	was to	 establish	 a	meaningful	

baseline	against	which	to	compare	the	results	of	the	subsequent	experimental	treatments	as	

described	in	Section	3.

To	establish	 these	baseline	estimates,	we	began	 this	 treatment	in	 the	same	manner	as the	

original	Welsh	et	al.	(1995)	survey—by	showing	respondents	a	picture	of	the	Grand	Canyon	

below	the	GCD and	informing	them	that	“the	survey they	you are	about	to	take is	part	of	a	

study	 of	 issues	 concerning	 operations	 of	 the	 Glen	 Canyon	 Dam.” We	 stressed	 that	

respondents’	 opinions	 on	 these	 issues	 were one	 factor	 that government	 officials would	

consider when	making decisions	on	the	operations	of	the	GCD.

After providing	 respondents	with	 this	general	overview	 of	 the	 study purpose,	we	 showed

them	a	series	of	pages	containing	information	relevant	to	operations	of	the	GCD.	Mirroring	

the	original	Welsh	et	al.	(1995)	study,	we	began	by	presenting background information	on	

the	GCD,	 the	 Glen	 Canyon	Study	Area,	 and	 how	 the	GCD affects	 the	 Colorado	River	in	 the	

Study	Area.	The	information	we	presented	was	identical	to	that	in	the	original	Welsh	et	al.	

(1995)	 study	 and	 included facts	 about	 the	 history,	 location,	 and	 purpose	 of	 the	 GCD,	

features	of	the	Study	Area	below	the	GCD,	and	how	operations	of	the	GCD affect	the	flow	of	

the	 Colorado	 River	 in	 the	 Study	 Area.	 	 To	 gauge respondents’	 comprehension	 of	 the	

information, we	 asked	 them	 two	 factual	 questions	 about	 what	 they	 had	 just	 read,	 an	

approach	we	employed after	each	page	of	information	presented	to	respondents (for	a	total	

of	six	factual	questions).
REPLICATION TREATMENT PAGE | 19

Again	 paralleling	 the	 structure,	 wording,	 and	 ordering	 of	 the	 original	Welsh	 et	 al.	 (1995)	

study,	we	next presented	 respondents	with	information	on	natural	 resources	in	 the	study	

area.	 Specifically,	 we	 provided	 information	 on	 archaeological	 sites	 near the	 river,	 the	

beaches	along	the	river,	and	the	status	of	trout	and	native	species	of	fish	in	the	river. This	

information	 was	 followed	 by	 a	 page	 that	 summarized	 concerns	 that	 “some	 people”	 have	

about	the	beaches,	archeological	sites,	and	fish	in	the	Study	Area. In	particular,	respondents	

were informed	 of	 concerns	 over	 shrinking	 beaches,	 the	 disruption	 of	 archeological	 sites,	

and	 the	 viability	 of	 fish.	 As	 with	 the	 other	 information	 provided	 to	 respondents,	 the	

concerns	presented	to	them	were drawn	from	the	original	Welsh	et	al.	(1995)	study.

The	final	page of	background	information	provided	to	respondents	states	that	changing	the	

operations	of	the	dam	could	address	some	of	the	concerns	about	the	natural	resources—the	

beaches,	archeological	sites,	and	fish—in	the	Study	Area.	Respondents	were	also	told	about	

some	 additional	 possible	 effects	 of	 changing	 dam	 operations,	including	increases	in	 some	

households’	 electric	 bills	 and	 higher	 costs	 for	 some	 farmers	 who	 use	 electricity	 to	 pump	

irrigation	water.

After presenting	respondents	with	all	of	this	information—the	background	information	on	

the	operations	of	the	GCD,	an	overview	of	natural	resources	in	the	Study	Area,	the	concerns	

over	 these	 resources,	 and	 potential	 effects	 of	 changing	 operations—we	 informed	

respondents	that	they	would be	presented	with	a	proposal	for	changing	dam	operations,	as	

well	as	a	description	of	 the	effects	of	 that	proposal, and	 that the	proposal	could	cost	 them	

money	in	 the	 form	 of	 higher	 taxes.	We	 then	 told	 respondents	 that they	would be	asked	if	

they	would	 vote	 “yes”	or	 “no”	on	 the	proposal,	both	if	 the	proposal	were	costless	 to	 them	

and	if	the	proposal	were	to	cost	them	some	positive	dollar	value. Drawing	on	the	Welsh	et	al.	

(1995)	study,	we	presented	respondents	with	reasons	why	they	might	vote	“yes”	or	“no”	on	

the	proposal.

We	 then	 presented	 respondents	 with	 the	 proposal	 for	 modifying	 dam	 operations,	 which	

would	 involve	 moderately	 reducing	 the	 daily	 fluctuations	 in	 the	 Colorado	 River	 level,	 as	

well	as	the	projected	effects	of	the	proposal, including:	

o A	stabilization	in	the	number	and	size	of	beaches;

o A	decrease	in	the	erosion	threat	to	archeological	sites;
REPLICATION TREATMENT PAGE | 20

o An	increase	in	the	area	available	for	vegetation;

o Improvements	in	the	conditions	for	fish;

o An	increase	in	the	electric	bill	for some	households;	and

o A	small	decline	in	income	for	300	farmers	in	southern	Utah.

Following this	 proposal	 we	 presented	 respondents	 with	 a	 question	 asking	 whether	 they	

would	support	the	proposal	if	its	passage	would	not	cost	them	anything.	Respondents	were

given	the	options	of	voting	“no,”	voting	“yes,”	or	stating	that	they	would	choose	not	to	vote.		

Respondents	were	then	informed	that	the	higher	electric	rates	accompanying	the	proposal	

would	not	be	able	to make	up	for	all	of	the	revenue	that	would	be	lost	under	the	proposal	

and	that	taxpayers	would	need	to	make	up	the	difference. Respondents	were	then	asked	if	

they	would	vote	for	the	proposal	if	its	adoption	would	cost	their	household	$25	in	increased	

taxes	every	year	for	the	foreseeable	future.	Respondents were	provided	with	the	following	

five	response	options:

o Definitely	vote	against	the	proposal

o Probably	vote	against	the	proposal

o Not	sure	whether	they	would	vote	for	the	proposal

o Probably	vote	for	the	proposal

o Definitely	vote	for	the	proposal

Table 3 presents	 the first	 set	 of results	 from	 the	 replication	 treatment.	Specifically, under	

the	 condition	 that	 passage	 of	 the	 proposal	 would	 cost	 respondents	 nothing, the	 table

presents the	 percentage of	 respondents	 who	 stated	 they	 would	 vote	 for	 the	 proposal	 to

change	dam	 operations,	 the	percentage	 stating	 that	 they	would	 vote	against	 the	proposal,	

and	 the	percentage	stating	 that	 they	would	choose	not	 to	vote.		As	a	point	of	comparison,	

Table	3	also	 presents	 the	analogous	 results	 from	 the	 original	Welsh	et	al.	 (1995)	 study—

results	from	when	respondents	were	asked	how	they	would	vote	if	passage	of	the	proposal	

would	not	cost	them	anything.
REPLICATION TREATMENT PAGE | 21

Table	3:	Replication	Treatment	Results—No	Cost

Our	Study Welsh	et	al.	(1995)	Study

Vote	for	proposal	to	

change	operations 70.6% 71%

Vote	against proposal	to	

change	operations 17.3% 17%

Would	not	vote 12.2% 12%

Table	3 makes	clear	that	our “no	cost” results	were remarkably	close to	the “no	cost” results	

as	presented	in	Welsh	et	al.	(1995),	with	71 percent of	respondents in	both	studies stating	

they	would	vote	for the	proposal	for	changing	dam	operations,	17 percent stating	that	they	

would	vote	against the	proposal,	and	12 percent stating	that	they	would	choose	not to vote.	

Although	 this	 treatment	 was	 not	 intended	 to	 reproduce	 the	 original	 Welsh	 et	 al.	 (1995)	

estimates,	 the fact	 that	 they	 are	 so	 similar	 is	 suggestive—although	 not	 definitive—of	

preferences	remaining	 fairly	stable	over	 two	decades	since	 the	Welsh	et	al.	 (1995)	survey	

was	 fielded.	Additionally,	 the	similarity	 of	 the	 results	suggests	 that	survey	mode	effects—

the	original	Welsh	et	al.	(1995)	was	fielded	by	mail	whereas	ours	was	Internet-based—are	

negligible.	Regardless,	the	results	presented	above	represent	a	valid	baseline	against	which	

the	results	of	the	additional experimental	treatments	can	be	compared.	

Table	 4 presents	 the	 second	 set	 of	 results	 from	 the	 replication	 treatment.	 In	 particular,	

under	the	condition	that	passage	of	the	proposal	would	cost	respondents	$25	in	increased	

taxes	each	year	 for	the	 foreseeable	 future,	Table	4 presents	the	percentage	of respondents

who	 stated	 they	 would	 vote	 for	 the	 proposal	 to	 change	 dam	 operations,	 the	 percentage	

stating	that	they	would	vote	against	the	proposal,	and	the	percentage	stating	that	they	were	

unsure	 or	 would	 choose	 not	 to	 vote. We	 note	 that	 these	 results	 are	 for	 all	 individuals	

assigned	to	the	replication	treatment,	not	just	those	who	stated	they	would	vote	to	change	

operations	at	zero	cost. Note	that the	original	Welsh	et	al.	(1995)	study	asked	respondents	

whether	 they	would	vote	for	 the	proposal	over	a	wide	range	of	dollar	costs.	For	 this	pilot	

study,	by	contrast, we	asked	respondents	only	 to	indicate	 their	vote	when	costs	would	be	

$25.	For	that	reason, Table	4 does	not	show	analogous	results	from	the	Welsh	et	al. (1995)	

study.
REPLICATION TREATMENT PAGE | 22

Table	4:	Replication	Treatment	Results—Cost	of	$25

Our	Study

Vote	for	proposal	to	change	operations 46.6%

Vote	against proposal	to	change	operations 32.7%

Not	sure/Would	not	vote 20.7%

The	 results	in	Table	4 demonstrate	 that	nearly	47	percent of	the	 respondents would	 vote	

for	 the	 proposal	 if	 it	 were	 to	 cost	 them	 $25	 in	 increased	 taxes	 annually.	 This	 number	 is	

substantially	lower	than	the	71	percent	of	respondents	who	would	vote	for	the	proposal	if	it	

were	to	cost	them	nothing.	Nearly	a	third of	the	respondents—33 percent—state	that	they	

would	vote	against the	proposal	if	its	passage	were	to	cost	them	$25, while	approximately	

20	percent	of	respondents	were	unsure	as	to	how	they	would	vote	or	would	choose	not	to	

vote.
BASELINE RANDOMIZED INFORMATION TREATMENT PAGE | 23

5. Baseline Randomized Information Treatment

As	Section	4 made	clear, the	replication	treatment	presented	respondents	with	a	substantial	

amount	 of	 information	 relevant	 to a	 potential	 change	 in GCD operations. In	 particular,	 it	

provided	them	with	background	information	on	the	operations	of	the	GCD,	an overview	of	

natural	resources	in	the	Study	Area,	the	concerns	over	these	resources,	and	potential	effects	

of	 changing	 operations.	 In	 order	 to maintain	 fidelity	 to	 the	 replication	 effort,	 all	 of	 this	

information	was	presented	in	 the same	order	as	it	was	in the	original	Welsh	et	al.	 (1995)	

study. Thus,	our first	post-replication	treatment involved	randomly	varying the	ordering	of	

information	presented	in	the	replication	treatment.		

The	purpose	of	this	treatment	is	two-fold.	First,	it	is	possible	that	the replication	results—

the	proportion	of	respondents	supporting	the	proposal	to	change	GCD operations—were	at	

least	partially	attributable	to	the	order	in	which information relevant	to	a	potential	change	

in	GCD operations was	presented,	which	tended to	highlight	the	ecological	dimensions.	This	

treatment	allows	us	 to	assess	 the	extent	 to	which	 this may	be	 the	case.	Second,	and more	

importantly,	 this	 treatment	 is	 necessary	 for	 valid	 administration	 of	 the	 additional	

experimental	 treatments	 we	describe below.	In	these	 treatments	 we introduce	 additional	

dimensions	 of	 value	 that	 have	 the	 potential	 to affect	 preferences	 and	 valuations	 for

changing	 GCD operations.	 We	 introduce	 these	 additional	 value	 dimensions	 by	 providing	

respondents	with	relevant	information	about them	as	part	of	 the	background information	

described	in	 the	previous	 section (See	Appendix	B	 for	wording	and	Appendix	C	 for	 visual	

reproduction).	In	 order	 to avoid	 bias stemming from	 the	 order in	 which the information	

about these	additional	dimensions	is	presented in	 relation	 to	 the	information	about other	

dimensions, all	 background	 information	 must	 be	 randomly	 ordered. This	 treatment	

provides the baseline	 against	 which	 the	 experimental	 treatments	 assessing	 the	 effect	 of	

additional	 value	 dimensions	 on	 valuations	 and	 preferences	 over	 GCD operations	 can	 be	

validly	compared.

To	 implement	 this	 treatment	 we	 began with	 the	 background	 information used	 in	 the	

replication	 treatment,	 which	 was described	 in	 Section 4.	 The	 content	 and	 ordering	 of	 all	
BASELINE RANDOMIZED INFORMATION TREATMENT PAGE | 24

background	information	can	be	 found	in	 the	survey	instrument in	Appendix	C.	Separately	

for	each	page	of	information—the	background	information	on	the	GCD and	Study	Area,	the	

overview	 of	 natural	 resources	 in	 the	 Study	 Area,	 the	 summary	 of	 concerns	 over	 those	

resources,	 and	 the	 description	 of	 how	 changing	 GCD operations	 might	 address	 some	

concerns—we	 grouped	 the	 text	into	 blocks	of	information on	 the	 same	 substantive	 topic.	

For	 example,	 on	 the	 page	 that	 overviews	 natural	 resources	 in	 the	 Study	 Area,	 one	

information	 block	 contains	 all	 text	 addressing	 beaches,	 the	 second	 information	 block	

contains	 all	 text	 describing	 the	 archeological	 sites,	 and	 the third	 block	 contains all	

information	relevant	to	fish.	The information groupings	for	all	pages	of	background	material

are	provided	in	Appendix	C.

Instead	 of	 encountering	 all	 background	 information	 in	 the	 precise	 ordering	 found	 in	 the	

original	Welsh	et	al.	(1995)	survey,	respondents	assigned	to	this	treatment were	exposed	to	

randomly	 ordered blocks	 of information across	 all	 pages	 of	 background	 material.	

Continuing	 to	 use	 the	 page	 that	 overviews	 natural	 resources	 in	 the	 Study	 Area	 as	 an

illustrative example,	one	respondent	might	see	 the	page in	a	 format where all fish-related	

information	is	at	the	top,	all	beach-related	information	is	in	the	middle,	and the description

of	archeological	sites is	at	the	bottom	of	the	page.	A	second	respondent	would	likely	see	the	

page	 in	 a	 completely	 different	 format—archeological	 information	 might	 be	 at	 the	 top,	

followed	by	fish-related	information	in	the	middle,	and	all	information	relevant	to	beaches	

at	the	bottom.

In	 addition	 to	 randomizing	 blocks	 of	 background	 information,	 this	 treatment	 also	

randomizes	 the	 order	 of	 the	 bulleted	 information	 in	 the	 proposal	 for	 changing	 dam	

operations.	For	randomization	purposes	in	the	proposal,	each	bullet	was	treated	as	its	own	

grouping. See	Appendices B and	C	for	the	specific	content	of	the	bulleted	information	in	the	

proposal.

Table	5 presents	 the	 results	 of	 the	 randomized	information treatment. It	presents	 results	

under	the	condition	where	passage	of	the	proposal	would	cost	respondents	nothing as	well	

as	under	the	condition	where	it	would	cost	respondents	$25	annually	in	additional	taxes	for	

the	foreseeable	future.	To	facilitate	comparison,	the	replication	treatment	results—for	both	

the	“no	cost” and	“cost	of	$25” conditions—are	also	presented. The	results	demonstrate	that	
BASELINE RANDOMIZED INFORMATION TREATMENT PAGE | 25

randomizing	 the	 order	 in	 which	 background	 information	 is	 presented,	 as	 well	 as	

randomizing	the	order	of	the	bulleted	information in	the	proposal,	has	neither	a	statistically	

significant	 nor	 substantively	 meaningful effect	 on	 respondent preferences	 for	 GCD

operations.	 These	 null	 effects	 are	 present	 for	 both	 the	 “no	 cost”	 and	 the	 “cost	 of	 $25”	

conditions.

Table	5:	Randomized	Information	Treatment	Results

No	Cost Cost	of	$25

Randomized	

Information	

Treatment

Replication	

Treatment

Randomized	

Information	

Treatment

Replication	

Treatment

Vote	for	proposal	to	

change	operations 71.0% 70.6% 42.5% 46.6%

Vote	against

proposal	to	change	

operations

17.3% 17.3% 33.7% 32.7%

Not	sure/Would	not	

vote 11.8% 12.2% 23.8% 20.7%

The	results	in	Table	5 have	important	implications.	In	particular,	the	results make	clear	that	

the	order	in	which	the	information	was	presented	in	the	replication	treatment	did	not	have	

an	 effect	 on	 respondents’	 expressed	 preferences	 for	 operations	 of	 the	 GCD.	 Although	

perhaps	surprising	from	a	substantive	standpoint,	this	result is appealing	from	a	statistical	

standpoint	 because	 it	 means	 we	 can	 validly	 combine	 respondents	 assigned	 to	 either	 the	

replication	 treatment	 or	 the	 randomized	 information	 treatment into	 a	 single group.	 This

increases the	sample	size	of our	baseline	treatment	group,	which	correspondingly	increases

the power	 to	 detect	 statistical	 differences	 between	 this	 baseline	 treatment	group	and	 our	

other	experimental	treatment	groups.	

However,	as	noted	above,	the	most	important	purpose	of	this	treatment	was	not	to	estimate	

the	 effect	 of	 information	 ordering	 on	 respondent	 valuations	 and	 preferences	 for	 GCD

operations,	 but	 rather	 to	 establish	 a	 valid	 baseline	 against	 which	 the	 additional	

experimental	 treatments	can	be	compared.	As	it	happens,	 the	effects	of	 this	 treatment	are

neither	statistically	nor	substantively	different	from	the effects	of	the replication	treatment,	

which	 allows	 us	 to	 use	 respondents	 assigned	 to	 either	 group	 as	 the	 baseline	 for	 future	

experimental	comparisons.
REFERENDUM OPTIONS TREATMENT PAGE | 26

6. Referendum Options Treatment

The	 original	 Welsh	 et	 al.	 (1995)	 survey,	 as	 well	 as	 our	 replication	 and	 randomized	

information	 treatments,	 asked	 respondents whether	 they	 would	 vote	 “yes” or	 “no” on	 a	

proposal	 for	 changing	 dam	 operations.	 Figure	 1 illustrates	 the	 structure of	 the	 proposal,	

which	first	describes	the	nature	of	the	change	in	dam	operations—a	moderate	reduction in	

the	daily	fluctuations in	the	Colorado	River	level—before	presenting	the	projected	effects	of	

the	change	in	operations	in	a	bulleted	format.

Figure	1:	Single	Proposal	Format	for	Valuation

The choice	 architecture underlying	 presentation	 of	 the	 proposal	 could	 have	 the	 effect	 of	

privileging the	 proposal.	 	 Specifically,	 the	 fact	 that	 respondents	 are	 presented	 with	 the	

proposal	 and	 then	 asked	 if	 they	 would	 support	 it	 renders	 it potentially	 susceptible	 to	

acquiescence	 bias,	 which	 is	 the	 tendency	 for	 individuals	 to	 agree	 with	 a	 statement	 or	

proposal	 regardless	 of	 its	 content.	 A	 substantial	 body	 of	 scholarly	 literature	 has	
REFERENDUM OPTIONS TREATMENT PAGE | 27

demonstrated	 the	 existence	 of	 acquiescence	 bias	 (see	 Krosnick	 and Presser	 (2010)	 for	 a	

review)	and	it	has	been	shown	to	be	of	particular	concern	in	“yes/no”	contexts	like	the	one	

underlying	 the	 proposal	 presented	 to	 respondents. Moreover,	 in	 the	 specific	 case	 of	 the	

proposal	for	changing	GCD operations,	the	respondents	may	question	why	experts	went	to	

the	expense	and	effort	of	constructing	the	proposal and	asking	their	opinion	about	it if	the	

proposal was	not	worthwhile.		

In	addition to	its potential	susceptibility	 to	acquiescence	bias,	 the current structure	of	 the	

proposal	prevents assessment	of	whether	some	respondents	may	actually	place	a	positive	

value	on	maintaining	current	dam	operations. As	it	was	administered	in	the	original	Welsh	

et	al.	(1995)	survey, as	well	as	in	our	replication	and	randomized	information	 treatments,	

respondents	 who	 state	 that	 they	 would	 vote	 against the	 proposal are automatically

assumed	 to	 value	 it at	 $0.	 The	 structure	 of	 the	 proposal	 does	 not	 render	 it	 possible	 for	

individuals	to	express	a	“negative”	valuation or	preference	for	the	proposal.	That	is,	there	is	

no	mechanism	for	individuals to	express	that	they	would	actively	prefer—be	willing	to	pay	

some	positive	dollar	amount—for	the	proposal	to	not go	into	effect	(i.e.	to	maintain	current	

dam	 operations).	 The	 lack	 of	 a	 mechanism	 for	 expressing	 such	 “negative”	 valuations	 is	

problematic	in	 cases	where	 the	good	 being	 valued	 has	effects	across	multiple	 dimensions	

and	 there	 is	 potential	 variation in	 whether	 respondents	 view	 these	 effects	 as	 positive	 or	

negative (see	 Bohara,	 Kerkvliet, & Berrens	(2001);	 Atkinson	 et	 al. (2004);	 and	 Loureiro,	

Loomis, &	Nahuelhual (2004)).	In	 the	case	of the	GCD as	a	major	component	of	a	coupled	

natural	 and	 human	 system,	 it	 is	 clear	 that	 changing	 operations	 will	 have	 effects	 across	

several	dimensions—ecological,	economic,	cultural,	and	others—and	it	is	likely that	some	of	

these	 effects	 would	 generally	 considered	 to	 be	 positive	 and	 others would	 be	 considered	

negative.	In	such	a	scenario,	precluding	respondents	from	expressing	a positive preference

or	 valuation for	 maintaining	 current	 operations	 will	 lead	 to	 biased	 inferences	 regarding	

respondents’	preferences	and	valuations	over	GCD operations.

Consequently, our	 third	experimental	 treatment	is	 designed	 to	address	 both	 the	 potential	

for	acquiescence	bias	and	the	lack	of	a	mechanism	for	expressing	positive	preferences	and	

valuations	 for	 maintaining	 current	 dam	 operations. In	 this	 treatment	 we	 provide	

respondents	 with	 the	 same	 background	 information	 as	 we	 did	 in	 the	 randomized	

information	 treatment.	 That	 is,	 all	 background	 information is	 presented using	 the	
REFERENDUM OPTIONS TREATMENT PAGE | 28

randomized	 information	 block	 approach	 described	 in	 Section 5. However,	 instead	 of	

eliciting	respondents’	preferences	and	valuations	for	dam	operations	using a	referendum	on	

a	 specific	 proposal—the	 approach	 used	 in	 Welsh	 et	 al.	 (1995)	 and	 the	 replication	 and	

randomized	 information	 treatments	 presented	 above—in	 this	 treatment	 we	 elicit	

respondents’	 preferences	 and	 valuations	 through a	 referendum	 on	 two	 options	 for	

operating	 the	 dam.	 As illustrated	 in	 Figure	 2 below,	 this	 approach	 to	 preference	 and	

valuation	elicitation presents	 respondents	with	 two	 options	 for	 operating	 the	 dam,	which	

we refer	to	as	Option	1	or	Option	2. Substantively,	the	two	options	are	maintaining current	

operations	or	changing	operations	in	a	manner	 that	moderately	reduces	daily	 fluctuations	

in	 the	 Colorado	 River	 level.	 To	 eliminate	 the	 threat	 of bias	 attributable	 to	 information	

ordering, we	 randomized	 whether	 maintaining	 current	 dam	 operations—and	 by	

implication	 changing	 dam	 operations—was	 listed	 as	 Option	 1 or	 Option	 2.	 The	

programming	underlying	 the	choice	 facing	 the	respondent	kept	 track	of	whether	Option	1	

referred	to	maintaining	current	dam	operations	or	changing	operations.	Figure	2	illustrates	

that	 the	 first	 row	 in	 each	 column	 describes the	 option	 for	 dam	 operations	 and	 each

subsequent	 row	 summarizes	 the	 effect	 of	 that option	 for	 dam	 operations	 along	 a	 given	

dimension of	value.	These	effects	are	generally	presented in	relation	to	the	other	option	for	

dam	operations.
REFERENDUM OPTIONS TREATMENT PAGE | 29

Figure	2:	Two-Option	Format	for	Valuation

After	 presenting	 respondents	 with	 the	 two	 options	 for	 dam	 operations, we	 described a	

situation	in	which	they	would	have	an	opportunity	to	vote	for	Option	1	or Option	2. We	then	

asked	them	whether	they	would	vote	 for	Option	1	or	Option	2 if	adoption	of	either	option

would	 not	 cost	 them	 anything.	Respondents	 could	 then	 select	Option	 1,	Option	 2,	 or	they	

could	indicate	that	they	would	choose	not	to	vote.
REFERENDUM OPTIONS TREATMENT PAGE | 30

In	order to	elicit	WTP,	it	is	necessary	 to	make	 the	valuation	questions	 for	 the	 two	options	

comparable.7 For	that	reason,	we	needed	to	present	the	“no	change”	option	as	one	that	(like	

the	change	option)	would	require	 that	 the	respondent	consider	 their	willingness	to	pay	to	

continue	 to	operate	 the	dam	as	it	had	been	in	 the	past.	To	accomplish	 this,	upon	selecting	

Option	1	or	Option	2 under	the	condition	of	“no	cost”,	we	brought	respondents	to	a	screen	

stating	that	the	option	they	chose	(either	change	or	no	change	in dam	operations)	was	more	

expensive	 to	 operate	 and	 that	 it	 would	 therefore	 increase	 costs to	 taxpayers.	 We	 then	

presented	respondents	with	a	question	asking	whether	they	would	vote	for	the	option	they	

had	preferred at “no-cost’’	even	if	its	adoption	would	cost	their	household	$25	in	increased	

taxes	 every	 year	 for	 the	 foreseeable	 future.	 Respondents	 could	 state	 that	 they	 would	

“definitely	vote	against	the	option,” “probably	vote	against	the	option,” were	“unsure	if	they	

would	vote	for	the	option,” “probably	vote	for	the	option,” or	“definitely	vote	for	the	option.”

There	are	several	appealing	features	of	this	approach	to	eliciting	preferences	and	valuations.	

First,	it	mitigates	 the	 potential	 for	acquiescence	 bias	 by	 requiring	 respondents	 to	actively	

choose	between	one	of	two	options	for	dam	operations,	as	opposed	to	stating	whether	they	

would	 or	 would	 not	 support	 a	 single	 proposal	 for	 changing	 dam	 operations.	 Second,	 it	

provides	respondents	with	a	mechanism for	expressing	a	negative preference	and valuation	

for	changing	dam	operations,	which	can	alternatively	be	framed	as	a	positive	preference	or	

valuation	for	maintaining	current	operations.	Moreover,	as	noted	above, this	mechanism	is	

presented	in	a	willingness to pay	framework,	as	opposed	to	willingness to accept,	which	is	

the	framework	that	most	previous	attempts	to	provide	a	mechanism	for	negative	valuation	

have	ultimately	employed.	Because	individuals	value	gains	and	losses	differently,	combining	

estimates	 generated	 from	 willingness to pay	 and	 willingness to accept	 approaches	 is	

generally	not	thought	to	be	a	valid	approach.	The treatment employed	in	this	study	avoids	

such	a	problem	because	valuations	 for both maintaining	and	changing	dam	operations	are

elicited	in	a	willingness to pay	framework.

The	design of	this	treatment	allows	us	to	perform	two	major	analyses.	First,	it	allows	us	to	

assess	whether	eliciting	respondents’	preferences	through	a	referendum	on	two	options	for	

operating	the	dam,	as	opposed	to	a	referendum	on	a	proposal	for	changing	dam	operations,	

																																																							 7 We	could	not,	for	example,	meaningfully	compare	WTP	for	changed	operations	with	a	“willingness	

to	accept”	(WTA)	compensation	for	keeping	dam	operations	unchanged.	
REFERENDUM OPTIONS TREATMENT PAGE | 31

has	 an	 effect	 on	 support	 for	 changing	 dam	 operations.	 We	 make	 this	 assessment	 by	

comparing	the	responses	of	 two	groups	of	individuals—those	assigned	 to	 the	 two	options	

treatment	 described	 here	 and	 those	 assigned	 to	 the	 randomized	 information	 treatment	

described	in	 the	 previous	 section—to	 the	 question	 of	whether	they prefer	maintaining	 or	

changing	dam	operations if	adoption	of	either	policy	was	guaranteed	to	cost	them	nothing.	

Second,	we	are	able	to	assess	the	extent	to	which	respondents	may	place a	positive	value	on	

maintaining	dam	operations.	To	make	 this	assessment,	we combine	respondents’	answers	

to	the	question	of	whether	they	prefer	maintaining	or	changing	with	their	responses	to	the	

question	of	whether	respondents	would	vote	for	the	option	they prefer	if	it	cost	them	$25 in	

additional	taxes	annually.	Combining	responses	to	these	two	questions allows	us	to	classify	

respondents	into	one	of	five	groups:

1. Prefer	maintaining	operations, and	willing	to	pay	$25	annually	to	do	so;

2. Prefer	maintaining	operations,	but	not	willing	to	pay	$25	annually	to	do	so;

3. Prefer	changing	operations, and	willing	to	pay	$25	annually	to	do	so;

4. Prefer	changing	operations,	but	not	willing	to	pay	$25	annually	to	do	so;	and

5. Would	choose	not	to	vote.

To	 the	 extent that	 respondents	 fall	into	 the	 first	 category—those	who	prefer	maintaining	

operations	and	are	willing	to	pay	$25	to	do	so—we	gain	a	preliminary	sense	of	how	many	

individuals	 hold	 a significant negative	 value	 for	 changing	 dam	 operations.	 In	 traditional	

contingent	 valuation	 studies,	 including	 the	 original	 Welsh	 et	 al.	 (1995)	 study,	 these	

individuals	would	be	assumed	to	hold	a	value	of	$0	for	changing	dam	operations. Through	

our	research	approach,	we	will	be	able	 to	assess	 the	extent	 to	which	 this	assumption	may	

lead	to	biased	willingness to pay	estimates.

Table	6 presents	the	results	of	the	“two	options”	treatment. The	numbers	in	bold	represent	

the	 results	 of	 the	 first	 major	 analysis—assessing	 the	 effect	 of	 eliciting	 respondents’	

preferences	 through	 a	 referendum	 on	 two	 options	 for	 dam	 operations,	 as	 opposed	 to	 a	

referendum	on	a	single	proposal	for	changing	operations.	Although	the	table indicates that	a	

slightly	lower	percentage of	respondents in	the	two	options	treatment	would	vote	to	change	

dam	operations—67.8	percent versus	70.6	percent in	the	replication	treatment—the	results	

of	a	chi-square	test	reveal	that	the	distribution	of	responses	in	the	two	treatments	are	not	
REFERENDUM OPTIONS TREATMENT PAGE | 32

statistically	 different	 from	 one	 another.

8 However	 the	 precision	 of	 these	 estimates	 can	 be	

increased	in	a	more	complete	model,	with	adequate	statistical	controls.	This	more	complete	

analysis	is	undertaken	in	Section	9.

Table	6:	Referendum	Options	Treatment	Results

Two	Options	Treatment Replication	Baseline

Vote	to	change	operations 67.8% 70.8%

					Would	pay	$25 46.9% N/A

Vote	to	maintain	operations 21.3% 17.3%

				 Would	pay	$25 6.9% N/A

Would	not	vote 11.0% 12.0%

As	noted earlier,	the	primary contribution	of	this	treatment	is	the	ability	to	assess	whether	

some	 fraction	 of	 respondents	 hold	 a	 positive	 value	 for	 maintaining	 dam	 operations.	 The	

results	presented	in	Table	6 indicate	that a non-trivial	percentage	of	respondents	do indeed	

hold	such	values.	Specifically,	Table	6 demonstrates	that	nearly	7	percent	of	respondents	in	

the	 two	 options	 treatment	would	 pay	 $25	 to	maintain	 current	 operations.	This	 7	 percent,	

which	is	statistically	different	from	zero,	represents	one-third	of	all	respondents	who	would	

vote	 to	 maintain	 operations.	 Such	 a	 finding	 is	 noteworthy	 because,	 as	 discussed earlier,

traditional	 contingent	 valuation	 studies	 typically	 assume	 these	 respondents	 to	 hold	 a	 $0	

value	 for	 changing	 dam	 operations.	 Our	 results	 demonstrate	 that	 such	 an	 assumption	 is	

incorrect, as	 these	 respondents	 effectively	 hold	 a	 “negative”	 WTP	 for	 changing	 dam	

operations,	which	implies	that	any	population	WTP	estimates for	changing	dam	operations

are	potentially	biased.

																																																							 8 The lack	of	a	statistical	difference	between	the	two	treatments	is	attributable in	part to	the	fact	that	

the	estimated	difference	between	the	two	treatment	groups	is	modest	in	magnitude—on	the	order	of	

3-4	percentage	points—coupled	with	the	fact	that	each	experimental	treatment	group	only	contains	

approximately	 350	 respondents.	A	 power	 analysis	 reveals	 that,	 for	 a	 simple	 chi-square	 test	 of	 this	

kind, each	 group	 would	 need	 to	 contain	 approximately	 3,700	 respondents	 in	 order	 to	 detect	 a	

difference	of	the	magnitude	reported	in	Table	6.
ADDITIONAL VALUE CONSIDERATION TREATMENTS PAGE | 33

7. Additional Value Consideration Treatments

The	 original	 Welsh	 et	 al.	 (1995)	 study	 provided	 respondents	 with	 information	 on	 value	

dimensions	 that	are	 undoubtedly	 relevant	 to	 decisions	 regarding GCD operations,	 namely	

the	 ecological	 and	 cultural	 dimensions.	 However,	 the	 1995	 study	 did	 not	 provide	

information	 on	 other	 dimensions	 that	 are	 potentially	 relevant	 to	 individuals’	 preferences	

and	valuations	of GCD operations.	Our	 fourth,	 fifth,	and	sixth	experimental	 treatments	are	

thus	devoted	 to	assessing	 the	effects	of	introducing	background	information	on	additional	

value	dimensions	and	considerations	of respondents’	expressed	preferences	and	valuations	

of	 GCD operations.	 Specifically,	 we	 introduce	 information	 and	 summarize	 the	 potential	

effects	 of	 changed	 dam	 operations	 on:	 (i)	 Native	 American	 tribes;	 (ii)	 Rural	 Western	

communities;	and	(iii)	both	Native	American	tribes	and	rural	Western	communities.9

As	 discussed	 in	 Section	 2,	 when	 evaluating	 the	 non-market	 value	 effects	 of	 changes	 in	

components	 of	 complex	 coupled	 human	 and	 natural	 systems,	 it	 is	 critical	 to	 take	 into	

account	 the	 manner	 in	 which	 the	 proposed	 changes	 will	 affect	 communities	 that	 have	

adapted	 to	 the	 prior	 system	 arrangements	 (Jacobs,	 2011;	 Kareiva	 et	 al.,	 2012;	 Bromley,	

2000).	 Loomis	 (2014)	 outlines	 the	 manner	 in	 which paternalistic	 altruism	 serves	 as	 the	

theoretical	 basis	 for	 including	 these	 additional	 value	 dimensions	 in	 a	 non-market	

framework.	Changing	dam	operations	is	projected	to	result	in	increased	electricity	costs	(or	

reduced	transfer	benefits)	for	many	Native American	tribes	and	rural	Western	communities,	

but	 these	 increased	 electricity	 costs	 per	 se are	 not	 a	 sufficient	 basis	 for	 including	 these	

additional	 value	dimensions—these	effects	can	be	 valued	in	a	market	 setting.	Rather,	it	is	

the	 respondents’	 desire	 for	 these	 communities	 to	 have	 continued	 access	 to	 cheaper	

electricity and	 related	 benefits—referred	 to	 as	 paternalistic	 altruism—that	 serves	 as	 the	

theoretical	 basis.	 This	 paternalistic	 altruism	 could	 stem	 from	 any	 number	 of	 sources,	

including	maintaining	a	unique	culture,	mitigating	threats	to	specific	occupations	or	way	of	

life,	or	others.

																																																							 9 The	question	wording	for	the	added	value	dimensions	was	pre-tested	in	on-line	applications	by	

researchers	and	students	in the	OU	Center	for	Energy,	Security	&	Society.	
ADDITIONAL VALUE CONSIDERATION TREATMENTS PAGE | 34

To	implement	these	treatments	we	began with	the	background	information	described	in	the	

randomized	 information	 treatment	 and	 structured	 the	 referendum	 on dam	 operations	 in	

the	 “two	 options”	 format	 described	 in	 the	 previous	 section.	 Then,	 on	 each	 page	 of	

background	information	presented	to	respondents—the	summary	of	operations	of	the	GCD

and	 status	 of	 the	 Study	 Area,	 the	 overview	 of	 natural	 resources	 in	 the	 Study	 Area,	 the	

concerns	 over	 these	 resources,	 and	 the	 potential	 effects	 of	 changing	 operations—we	

inserted	language	 relevant	 to	 the	additional	value	 dimensions	 covered by	each	 respective	

treatment.10 We	also	inserted	language	relevant	to	the	additional	value	dimensions	into	the	

referendum	 on dam	 operations	 faced	 by	 respondents. Below	 we	 present	 the	 specific	

language	inserted	 for	each	 treatment.	A	 visual	presentation	 of	each	 of	 these	 treatments	is	

provided	in	Appendix	C.

Although	 most	 of	 the	 insertions	 and	 modifications	 are	 treatment-specific—we	 describe	

these	treatment-specific	changes	in	respective	sections	below—there	are	some	changes	that	

are	applicable	to	all	three	experimental	treatments	described	in	this	section.		These	include:

o Changing	 the	 purpose	 of	 the	 study	 presented	 to	 respondents	 from	 “This	 study	

deals	with	the	ways	Glen	Canyon	Dam	could	be	operated	to	benefit	natural	and	

cultural	resources	in	the	Study	Area”	to	“This	study	deals	with	options	for	how	

the	Glen	Canyon	Dam	could	be	operated	 that	would	affect	natural	resources	in	

the	 Study	 Area	 and	 the	 communities	 that	 have	 relied	 on	 the	 hydropower	

produced	by	the	Glen	Canyon	Dam.”

o Changing	 the	 title	 of	 the	 page	 presenting	 the	 overview	 of	 natural	 resources	in	

the	study	area	from	“Natural	Resources	in	the	Study	Area”	to	“Natural	Resources	

in	the	Study	Area	and	Rural	Western	Communities	in	the	Region.”

o Changing	 the	 text	 introducing	 the	 natural	 resources	 page	 from	 “The	 natural	

resources	in	 the	Study	Area	are	located	in	and	along	 the	Colorado	River	below	

Glen	Canyon	Dam”	to	“The	natural	resources	in	the	Study	Area	are	located	in	and	

along	 the	 Colorado	 River	 below	 Glen	 Canyon	 Dam.	 Many	 of	 the	 small	

communities	that	have	relied	on	hydropower	from	the	Glen	Canyon	Dam	are	in	

rural	parts	of	Arizona,	Colorado,	New	Mexico	and	Utah.”

																																																							 10 The	content	of	the	added	value	considerations	was	based	on	extensive	discussions	with	experts	on	

the	relevant	issues,	resulting	in	iteratively	refined	information	wording.
ADDITIONAL VALUE CONSIDERATION TREATMENTS PAGE | 35

o Changing	 the	 title	 of	 the	 page	 summarizing	 concerns	 from	 “Some	 People	 are	

Concerned	About	These	Resources”	to	“Some	People	are	Concerned	About	These	

Resources	and	Communities.”

o Changing	the	title	of	the	page describing	the	effects	of	changing	dam	operations	

from	 “Scientists	have	learned	 that	by	changing	 the	way	water	is	released	 from	

the	 dam,	 primarily	 by	 reducing	 the	 size	 of	 daily	 fluctuations,	 some	 of	 the	

concerns	about	 the	natural	resources	in	 the	Study	Area	could	be	addressed”	 to	

“Scientists	 have	 learned	 that	 by	 changing	 the	 way	 water	 is	 released	 from	 the	

dam,	natural	resources	in	the	Study	Area	and	residents	of	the	surrounding	rural	

communities	could	both	be	affected	in	a	variety	of	ways.”

All	 remaining	 changes	 are	 treatment-specific,	 and	 we	 describe	 them	 in	 their	 respective	

subsections	below.

7.1 Native American Tribes

For	 the	 treatment	 introducing	 Native	 American	 tribes	 as	 a	 value	 dimension	 relevant	 to	

respondents’	 preferences	for	and	 valuations	 of dam	 operations,	we	inserted	 the	 following	

information	 block—with	 randomized	 placement	 along	 with existing	information	 blocks—

into	the	page	overviewing	natural	resources	in	the	study	area:

o Over	 50	 American	 Indian	 Tribes	 receive	 allocations	 of	 electricity	 from	 the	

hydropower	produced	by	the	Glen	Canyon	Dam.	

• Included	are	the	Navaho,	Utes,	Apache,	Pueblo	Indian	communities,	and	

many	other	tribes.

o These	allocations	provide	low	cost	and	stable	electric	power	to	the	tribes,	or	are	

used	to	offset	more	costly	electricity	purchased	from	area	utilities.

o The	benefits	to	the	tribes	help	tribal	residential	users	and	businesses,	and	are	an	

important	part	of	sustaining	tribal	communities	and	improving	their	economies.

For	 the	 page	 summarizing	 the	 concerns	 that	 people	 have	 about	 natural	 resources	 and	

communities	 in	 the	 study	 area,	 we	 inserted	 the	 following	 information	 block,	 which	 was	

again	placed	in	randomized	order	among all	existing	information	blocks:
ADDITIONAL VALUE CONSIDERATION TREATMENTS PAGE | 36

o Many	Native	American	Tribes	rely	on	the	low	and	stable	prices	of	the	electricity	

produced	at	the	Glen	Canyon	Dam.

• Over	 50	 American	 Indian	 Tribes	 receive	 economic	 benefits	 from	

hydropower	produced	at	the	Dam.

o Tribal	businesses	have	relied	on	the	low	and	stable	electricity	prices	in	creating	

and	maintaining	area	employment.

o For	 many	 Tribal	 residences,	 electricity	 produced	 at	 the	 Glen	 Canyon	 Dam	 is	

more	 affordable	 than	 electricity	 produced	 at	 power	 plants	 that	 rely	 on	 fossil	

fuels.

o The	 low	 and	 stable	 price	 of	 electricity	 produced	 at	 the	 Glen	 Canyon	 Dam	 has	

contributed	to	the stability	and	growth	of	communities	in	Tribal	areas.

On	 the	 final	 page	 of	 background	 information—describing	 how	 changing	 dam	 operations	

might	 affect	 natural	 resources	 and	 rural	 communities—we	 inserted	 the	 following	

information	 block,	 which	 was	placed	in	 random	 order along	 with	 all	 existing	information	

blocks:	

o Changing	the	way	water	is	released	through	the	dam	will	reduce	the	benefits	of	

low	 cost	 electricity	 from	 the	 Glen	 Canyon	 Dam	 to	 over	 50	 American	 Indian	

Tribes.

o Some	 Tribes	 would	 need	 to	 replace	 the low-cost	 electricity	 from	 the	 Glen	

Canyon	Dam	with	power	from	local	utilities.

• The	 cost	 of	 electricity	 from	 local	 utilities	 would	 be	 higher	 and	 more	

variable

• Higher	 and	 more	 uncertain	 prices	 for	 electricity	 would	 make	 it	 more	

difficult	for	business	to	compete,	thereby	reducing	jobs

• Some	 communities	in	 Tribal	 areas	may	 decline	 as	 a	 result,	 as	jobs	 and	

incomes	are	reduced	and	residents	migrate	to	other	areas

Finally,	 we	 inserted	 the	 following	 information	 block	 into	 the	 “two	 options”	 referendum	

facing	respondents.	As	is	the	case	with	all	other	insertions,	this	one	was	placed	in	random	

order along	 with	 all	 existing	information	 blocks.	 	 In	 the	 example	 below,	“Option	 1” is	for	

Changing	Dam	Operations	while	“Option	2” is	for	Continued	Dam	Operations.	This	is	merely

illustrative,	however,	and	the	reverse	order	could	also	be	the	case for	some	respondents.
ADDITIONAL VALUE CONSIDERATION TREATMENTS PAGE | 37

Option	1- Changed	operations Option	2- Continued	operations

Reduced	benefits	from	hydropower	for	

over	50	Native	American	Tribes.		These	

reduced	benefits	could	pose	a	threat	to	the	

communities	in	Tribal	areas	as	jobs	are	

reduced	and	residents	seek	employment	

elsewhere

Continued	benefits	from	hydropower	

production	for	over	50	Native	American	

Tribes.		These	continued	benefits	would	

sustain	jobs	and	support	communities	in	

Tribal	areas.

Table	 7 below	 presents	 the	 results	 of	 the	 treatment	 introducing	 implications	 for	 Native	

American	 tribes	 as	 a	 value	 dimension	 relevant	 to	 respondents’	 preferences	 for	 and	

valuations	of	dam	operations.	The treatment	results	are	presented	alongside	the	results	for	

the	“Two	Options	Referendum” treatment,	which	 serves	as	 the	most	appropriate	 baseline	

for	 purposes	 of	 comparison.	 The	 results	 in	 Table	 7 demonstrate	 that	 the	 addition	 of	

implications	 for	 Native	 American	 tribes has	 an	 effect	 on	 preferences	 for	 changing	 dam	

operations	 that	is	both	statistically	significant	and	substantively	large.	Whereas	over	 twothirds	 of	 respondents	 in	 the	 baseline	 condition	 stated	 that	 they	 would	 vote	 to	 change	

operations—and	nearly	half	of	all	respondents	stated	that	they	would	be	willing	to	pay	$25	

to	 do	 so—less	 than	 half	 of	 respondents	 assigned	 to	the	 new	 treatment	 support	 changing	

operations,	and	only	one-third	would	pay	$25	to	do	so.	Correspondingly,	the	proportion	of	

respondents	 who	 would	 vote	 to	 maintain	 operations	 increases	 from	 21	 percent	 in	 the	

baseline	 condition	 to	 nearly	 40	 percent	 in	 the	 new	 treatment;	 the	 percentage	 of	

respondents	 who	 would	 pay	 $25	 to	 maintain	 operations	 increases	 from	 7	 percent	 in	 the	

baseline	condition	to	17	percent	in	the	new treatment.	Clearly,	introduction	of	the	potential	

effects	 on	 Native	 American	 tribes	 as	 a	 value	 dimension	 has	 a	 substantial	 effect	 on	

respondents’	preferences	for	and	valuations	of dam	operations.

Table	7:	Native	American	Treatment	Results

Treatment	with	

Implications	for	Native	

Americans

Two	Options	Referendum

Vote	to	change	operations 49.5% 67.8%

		Would	pay	$25 33% 46.9%

Vote	to	maintain	operations 39.2% 21.3%

		Would	pay	$25 16.5% 6.9%

Would	not	vote 11.4% 11.0%
ADDITIONAL VALUE CONSIDERATION TREATMENTS PAGE | 38

7.2 Rural Western Communities

For	the	treatment	introducing	rural	Western	communities	as	a	value	dimension	relevant	to	

respondents’	 preferences	for	and	 valuations	of dam	 operations,	we	inserted	 the	 following	

information	 block—randomized with	 existing	 information	 blocks—into	 the	 page	

overviewing	natural	resources	in	the	study	area:

o Many	rural	areas	and	small	communities	in	Arizona,	Colorado,	New	Mexico,	and	

Utah	rely	on	hydropower	produced	at	the	Glen	Canyon	Dam.	The	economies	of	

these	communities	depend	heavily	on	ranching	and	farming.

o Hydropower	provides	low	cost	and	stable	power	to	farmers	and	ranchers.

• These	benefits	help	sustain	the	economies	and	ways	of	life	in	these	rural	

communities.

For	 the	 page	 summarizing	 the	 concerns	 that	 people	 have	 about	 natural	 resources	 and	

communities	 in	 the	 study	 area,	 we	 inserted	 the	 following	 information	 block,	 which	 was	

again	placed	in	randomized	order	among the existing	information	blocks:

o Rural	ranching and	farming	communities	in	Colorado,	New	Mexico,	Arizona,	and	

Utah	 rely	 on	 the	 low	 and	 stable	 prices	 of	 the	 electricity	 produced	 at	 the	 Glen	

Canyon	Dam.

• Many	 farmers	and	ranchers	in	these	communities	rely	on	electricity	 for	

pumping	water	for	irrigation

• Farmers	and	ranchers,	and	the	businesses	that	support	them,	have	relied	

on	the	low	and	stable	electricity	prices	in	creating	and	maintaining	area	

employment.

• For	many	rural	farming	and	ranching	communities,	electricity	produced	

at	the	Glen	Canyon	Dam	is	more affordable	than	electricity	produced	at	

power	plants	that	rely	on	fossil	fuels.

On	 the	 final	 page	 of	 background	 information—describing	 how	 changing	 dam	 operations	

might	 affect	 natural	 resources	 and	 rural	 communities—we	 inserted	 the	 following	

information	 block,	 which	 was	 again	 placed	 in	 randomized	 order	 among the existing	

information	blocks:	
ADDITIONAL VALUE CONSIDERATION TREATMENTS PAGE | 39

o Changes	in	 the	way	 the	 Glen	 Canyon	Dam	is	 operated	 are	likely	 to	 reduce	 the	

economic	well	being	of	some	of	the	small	ranching	and	farming	communities	in	

the	region.	

• Increased	costs	of	electricity	will	limit	 the	ability	of	some	ranchers	and	

farmers	to	irrigate	crops	and	maintain	livestock

• These	 changes	 may	 lead	 to	 reductions	 in	 employment	 and	 reduced	

income	for	businesses	in	rural	areas	that	serve	ranchers	and	farmers,	

• As	 jobs	 diminish,	 some	 of	 these	 small	 rural	 ranching	 and	 farming	

communities	will	decline	as	residents	seek	employment	elsewhere.

Finally,	 we	 inserted	 the	 following	 information	 block	 into	 the	 “two	 options”	 referendum	

facing	 respondents.	 As	 is	 the	 case	 with	 all	 other	 insertions,	 this	 one	 was	 placed	 in	

randomized	order	along	with	the	existing	information	blocks.	In	the	example	below,	“Option	

1” is	for	Changing	Dam	Operations	while	“Option	2” is	for	Continued	Dam	Operations.	This	

is	just	illustrative,	however,	and	the reverse	could	also	be	the	case	for	some	respondents.

Option	1- Changed	operations Option	2- Continued	operations

Higher	and	more	variable	electricity	costs	

for	some	farmers,	ranchers,	and	individuals	

living	in	small	western communities.	In	

some	of	these	communities,	increased	costs	

could	result	in	lost	jobs	and	increased	

migration	of	residents	to	other	areas.

Continued	low	and	stable	electricity	prices	

for	farmers,	ranchers,	and	individuals	

living in	small	western	communities.	These	

continued	low	and	stable	electricity	prices	

would	sustain	jobs,	supporting	populations	

in	small	farming	and	ranching	

communities.

Table	8 presents	 the	results	of	 the	 treatment	introducing	rural	Western	communities	as	a	

value	dimension	relevant	to	respondents’	preferences	for	and	valuations	of dam	operations.		

The	rural	Western	community	treatment	results	are	presented	alongside	the	results	for	the	

“Two	Options	 Referendum” treatment,	 which	 serves	 as	 the	most	 appropriate	 baseline	 for	

purposes	of	comparison.	

The	results	in	Table	8 demonstrate	that—similar	to	the	treatment including	implications	for	

Native	 American	 tribes—the	 rural	 Western	 community	 treatment	 has	 an	 effect	 on	

preferences	 for	 changing	 dam	 operations	 that	 is	 both	 statistically	 significant	 and	

substantively	 large.	 Whereas	 over	 two-thirds	 of	 respondents	 in	 the	 baseline	 condition	

stated	that	they	would	vote	to	change	operations—and	nearly	half	of	all	respondents	stated	
ADDITIONAL VALUE CONSIDERATION TREATMENTS PAGE | 40

that	they	would	be	willing	to	pay	$25	to	do	so—less	than	half	of	respondents	assigned	to	the	

rural	Western	community	treatment	support	changing	operations,	and	only	about	one-third	

would	pay	$25	to	do	so.	Correspondingly,	the	proportion	of	respondents	who	would	vote	to	

maintain	operations	increases	from	21	percent	in	the	baseline	condition	to	over	37	percent	

in	the	rural	Western	community	treatment;	the	percentage	of	respondents	who	would	pay	

$25	to	maintain	operations	increases	from	7	percent	in	the	baseline	condition	to	16	percent	

in	 the	 rural	 Western	 Community	 treatment.	 	 Clearly,	 introducing	 the	 effects	 on	 rural	

Western	 communities	 as	 a	 value	 dimension	 has	 a	 substantial	 effect	 on	 respondents’	

preferences	for	and	valuations	of dam	operations.

Table	8:	Rural	Western	Community	Treatment	Results

Rural	Western	

Community	Treatment Two	Options	Referendum

Vote	to	change	operations 49.1% 67.8%

		Would	pay	$25 33.9% 46.9%

Vote	to	maintain	operations 37.1% 21.3%

		Would	pay	$25 16.4% 6.9%

Would	not	vote 13.8% 11.0%

7.3 Native American Tribes + Rural Western Communities

Our	third	and	final	treatment	with	additional	value	considerations introduces	the	potential	

effects	on	both	Native	American	tribes	and	rural	Western	communities	as	value	dimensions	

relevant	 to	 respondents’	 preferences	 for	 and	 valuations	 of dam	 operations.	In	 effect,	 our	

third	treatment	combines	the	potential	effects	on	Native	American	tribes	and	rural	Western	

communities into	 a	 single	 treatment.	 We	 implement	 this	 treatment	 by	 exposing	

respondents	to	all	of	the	information	blocks describing	potential	effects	on Native	American	

and	rural	Western	communities,	as	described	above	in	subsections	7.1	and	7.2.

Table	9 presents	the	results	of	the	treatment	introducing	both	Native	American	tribes	and	

rural	Western	 communities	 as	 value	 dimensions	 relevant	 to	 respondents’	 preferences	for	

and	 valuations	of	dam	 operations.	The	 results	 are	 presented	 alongside	 the	 results	 for	 the	

“Two	Options	 Referendum” treatment,	 which	 serves	 as	 the	most	 appropriate	 baseline	 for	

purposes	of	comparison.	In	addition,	for	further	comparison,	the	respective	results	from	the	

treatments	 for	potential	effects	on	Native	American	tribes	and	rural	Western	communities

are	also	presented.	
ADDITIONAL VALUE CONSIDERATION TREATMENTS PAGE | 41

Table	9:	Native	American	+	Rural	Western	Community	Treatment	Results

Native	

American	+	

Rural	

Western	

Community	

Treatment

Two	Options	

Referendum

Native	

American	

Treatment

Rural	

Western	

Community	

Treatment

Vote	to	change	operations 32.4% 67.8% 49.5% 49.1%

		Would	pay	$25 21.8% 46.9% 33.0% 33.9

Vote	to	maintain	operations 57.1% 21.3% 39.2% 37.1%

		Would	pay	$25 30.3% 6.9% 16.5% 16.4%

Would	not	vote 10.5% 11.0% 11.4% 13.8%

The	results	in	Table	9 demonstrate	that	combining	the	Native	American	and	rural	Western	

community	 treatments	 has	 a	 cumulative effect	 on	 preferences	 for	 dam	 operations	 that	 is	

larger	than	the	effect	of	either	treatment	by	itself.	Specifically,	Table	9 reveals	that	less	than	

one-third	 of	 respondents	 assigned	 to	 the	 Native	 American	 +	 rural	 Western	 community	

treatment	would	 vote	 to	 change	 operations,	a	 smaller	 number	 than	 the	approximately	 50	

percent	of	respondents	who	would	do	so	when	each	treatment	was	presented	by	itself,	and	

a	substantially	smaller	number	than	the	68	percent	of	respondents	who	would	cast	such	a	

vote	 when	 neither	 value	 dimension	 is	 introduced.	 When	 the	 Native	 American	 and	 rural	

Western	 community	 treatments	 are	 combined,	 a	 solid	 majority	 of	 respondents—57	

percent—would	vote	to	maintain	operations	and	over	30	percent	of	respondents	would	pay	

$25	 to	 do	 so;	 less	 than	 one-third	 of	 respondents	 would	 vote	 to	 change	 operations.	 The	

pattern	of	results	shown	in	Table	9 demonstrates that	the	inclusion	of	the	value	dimensions	

concerning	 Native	 American	 tribes	 and	 rural	 Western	 communities operate	 as	

complements,	rather	than	substitutes,	for	one	another.
HYPOTHETICAL AIR EMISSIONS TREATMENT PAGE | 42

8. Hypothetical Air Emissions Treatment

Changes	in	 the	operation	of	dams	with	hydropower	generators,	such	as	 the	GCD,	have the	

potential	to	substantially	change	levels	of	harmful	air	emissions	when reduced	(increased)	

quantities	of hydropower	lead	to	increased	(decreased)	demand	for	electricity generated	by	

burning	 fossil	 fuels.	 Thus,	 our	 final	 experimental	 treatment	 is	 designed	 to	 assess	 how	

changes	 in	 air	 emissions	 resulting	 from	 changes	 in	 dam	 operations	 would	 affect

respondents’	 preferences for	 and	 valuations	 of	 hydropower.	 Unlike	 the	 Native	 American	

and	 rural	 Western	 community	 treatments,	 the	 air	 emissions	 treatment	 is	 completely	

hypothetical—the	 changes	 currently	 under	 consideration	 for operations	 of	 the	 GCD are	

unlikely	to have	the	effects	we	describe	in	the	treatment—but	GCD operations	provides	an	

ideal	context	for	assessing	respondents’	preferences	for	and	valuations	of	the	dimension	of	

air	 quality.	 Because	 of	 the	 hypothetical	 nature	 of	 the	 treatment, we	 do	 not	 combine	 this	

treatment	 with	 the	 treatments	 that	 include	 effects	 on	 Native	 American	 tribes	 or	 rural	

Western	communities,	which	were presented	in	Section	7.

Although	this	treatment	is	unlikely	to	be germane	to	the	changes	currently	being	considered	

for	operations	of	 the	GCD,	it	is	 relevant	 to	administrative	and	policy	decisions	concerning

the	 addition	 or	 removal	 of	 the	 generators	 and/or dams	 that	 provide	 hydropower.	 This	

relevance	comes	 from	 the	 fact	 that	altering	hydropower	capacity	will	likely	affect	demand	

for	 electricity	 from	 conventional	 fossil	 fuel	 generators,	 and	 the	 associated	 levels	 of	 air	

emissions.	 For	 example,	 reducing	 hydropower	 capacity	 will	 require	 that	 electricity	 be	

replaced	 with power	 from	 some	 other	 source.	 To	 the	 extent	 that	 the	 replacement	 power	

comes	 from	 electricity	 generated	 with	 fossil	 fuels,	 the	 reduced	 hydropower	 capacity	 will	

have	 the	 net	 effect	 of	 increasing	 harmful	 air	 emissions.	 Similarly,	 increasing	 hydropower	

capacity	will replace	power	that	may otherwise	have	been	generated	by	fossil	fuels	and,	in	

doing	so,	decrease harmful	air	emissions.	By	gauging	how	respondents	value	the	“clean	air”	

characteristic	 of	 hydropower, the results	 described	 in	 this	 section	 are	 relevant	 to	 any	

decision	concerning increasing	or	decreasing	hydropower	capacity.

We	implement	 this	 treatment	in	much	 the	 same	 way	 as	we	 did	 the	Native	 American	 and	

rural	Western	community	treatments—we provide	respondents	with	information	blocks	on	
HYPOTHETICAL AIR EMISSIONS TREATMENT PAGE | 43

the	 topic	 that	 are	 placed	 in	 randomized	 order	 alongside	 existing	 information	 blocks	

described	 in	 Section	 6	 of	 this	 report.	 Below	 we	 detail	 the	 specific	 changes	 we	 made	 to	

implement	our	air	emissions	treatment.

o Changing	 the	 purpose	 of	 the	 study	 presented	 to	 respondents	 from	 “This	 study	

deals	with	the	ways	Glen	Canyon	Dam	could	be	operated	to	benefit	natural	and	

cultural	 resources	in	 the	 Study	 Area”	 to	 “This	 study	 deals	 with	 the	 ways	 Glen	

Canyon	Dam	could	be	operated	to	benefit	natural	and	cultural	resources	in	the	

Study	Area.	 It	also	concerns	 the	possible	effects	of	switching	 from	hydropower	

produced	at	the	dam	to	electricity	produced	at	conventional	power	plants	using	

fossil	fuels.”

o Changing	 the	 title	 of	 the	 page	 presenting	 the	 overview	 of	 natural	 resources	in	

the	study	area	from	“Natural	Resources	in	the	Study	Area”	to	“Natural	Resources	

in	the	Study	Area	and	Rural	Western	Communities	in	the	Region.”

o Changing	 the	 text	 introducing	 the	 natural	 resources	 page	 from	 “The	 natural	

resources	in	 the	Study	Area	are	located	in	and	along	 the	Colorado	River	below	

Glen	Canyon	Dam”	to	“The	natural	resources	in	the	Study	Area	are	located	in	and	

along	 the	 Colorado	 River	 below	 Glen	 Canyon	 Dam.	 Many	 of	 the	 small	

communities	that	have	relied	on	hydropower	from	the	Glen	Canyon	Dam	are	in	

rural	parts	of	Arizona,	Colorado,	New	Mexico	and	Utah.”

o Changing	 the	 title	 of	 the	 page	 summarizing	 concerns	 from	 “Some	 People	 are	

Concerned	About	These	Resources”	to	“Some	People	are	Concerned	About These	

Resources	and	Communities.”

o Changing	the	title	of	the	page	describing	the	effects	of	changing	dam	operations	

from	 “Scientists	have	learned	 that	by	changing	 the	way	water	is	released	 from	

the	 dam,	 primarily	 by	 reducing	 the	 size	 of	 daily	 fluctuations, some	 of	 the	

concerns	about	 the	natural	resources	in	 the	Study	Area	could	be	addressed”	 to	

“Scientists	 have	 learned	 that	 by	 changing	 the	 way	 water	 is	 released	 from	 the	

dam,	natural	resources	in	the	Study	Area	and	residents	of	the	surrounding	rural	

communities	could	both	be	affected	in	a	variety	of	ways.”

For	the	treatment	introducing	air	emissions	as	a	value	dimension	relevant	to	respondents’	

preferences	 for	 and	 valuations	 of dam	 operations,	 we	 inserted	 the	 following	 information	
HYPOTHETICAL AIR EMISSIONS TREATMENT PAGE | 44

block—placed	 in	 randomized	 order	 among existing	 information	 blocks—into	 the	 page	

overviewing	natural	resources	in	the	Study	Area:

o For	over	50	years,	hydropower	produced	at	the	Glen	Canyon	Dam	has	reduced	

the	need	 to	 rely	 on	electricity	produced	at	conventional	power	plants	 that	use

coal,	oil	or	natural	gas.

o By	 replacing	 fossil	 fuels,	 the	 hydropower	 produced	 at	 the	 dam	 has	 helped	

reduce	air	pollution	in	Colorado,	New	Mexico,	Arizona,	and	Utah.

o Over	 the	 history	 of	 the	 operations	 of	 the	 dam,	 hydropower	 has	 provided	

electricity	 without	 contributing	 to	 greenhouse	 gas	 emissions	 that	 cause	 the	

climate	to	change.

For	 the	 page	 summarizing	 the	 concerns	 that	 people	 have	 about	 natural	 resources	 and	

communities	 in	 the	 study	 area,	 we	 inserted	 the	 following	 information	 block,	 which	 was	

again	randomized	with	all	existing	information	blocks:

o For	 over	 50	 years,	 the	 hydropower	 produced	 at	 the	 Glen	 Canyon	 Dam	 has	

reduced	reliance	on	conventional	power	plants	using	fossil	fuels	like	coal,	oil	or	

natural	gas.

• Reliance	on	hydropower	instead	of	fossil	fuels	has	reduced	air	pollution.

• Reduced	 air	 pollution	 has	 helped	 preserve	 the	 scenic	 beauty	 of	 the	

Grand	Canyon.

• Reduced	air	pollution	has	helped	improve	the	health	of	residents	in	the	

region,	who	breathe	cleaner	air.

• Using	 hydropower	 instead	 of	 fossil	 fuels	 has	 resulted	 in	 lower	 overall	

emissions	of	greenhouse	gases	that	cause	climate	change.	

On	 the	 final	 page	 of	 background	 information—describing	 how	 changing	 dam	 operations	

might	 affect	 natural	 resources	 and	 rural	 communities—we	 inserted	 the	 following	

information	 block,	 which	 was	 placed	 in	 randomized	 order	 among	 the	 other information	

blocks:	

o Changes	in	the	way	the	Glen	Canyon	Dam	is	operated	will	reduce	the	amount	of	

hydropower	 produced,	 leading	 to	 greater	 use	 of	 electricity	 from	 conventional	

power	plants	using	coal,	oil	and	natural	gas.
HYPOTHETICAL AIR EMISSIONS TREATMENT PAGE | 45

• Air	 quality	 in	 the	 region	 will	 decline	 as	 pollution	 from	 conventional	

power	plants	increases.

• The	 health	 of	 the	 local	 populations	 may	 be	 negatively	 affected	 by	 the	

increased	 air	 pollution,	 which	 contributes	 to	 a	 number	 of	 serious	

diseases.

• Visibility	in	 the	 Study	 Area,	including	 the	 Grand	 Canyon,	may	 diminish	

due	to	increased	air	pollution.

• Greenhouse	 gas	 emissions	 will	 increase,	 as	 use	 of	 hydropower	 is	

replaced	by	use	of	electricity	from	burning	fossil	fuels.

• Adding	 to	 the	 amount	 of	 greenhouse	 gas	 emissions	 will	 contribute	 to	

changes	in	the	global	climate.

Finally,	 we	 inserted	 the	 following	 information	 block	 into	 the	 “two	 options”	 referendum	

facing	 respondents.	 As	 is	 the	 case	 with	 all	 other	 insertions,	 this	 one	 was	 placed	 in	

randomized	order	 among	 the	 other	information	 blocks.	In	 the	 example	 below,	“Option	 1”	

represents Changing	 Dam	 Operations, while	 “Option	 2”	 represents Continued	 Dam	

Operations.		This	is	just	illustrative,	however,	and	the	reverse	could	also	be	the case. The	full	

presentation	of	this	treatment,	as	seen	by	survey	respondents,	is	shown	in	Appendix	C.

Option	1- Changed	operations Option	2- Continued	operations

Increase	in	the	amount	of	air	pollutants,	

including	greenhouse	gases,	emitted	into	

the	atmosphere	in	the	region	near	the	dam.	

By	decreasing	air	quality,	these	pollutants	

will	likely	negatively	affect	the	visibility	in	

the	area	near	the	Grand	Canyon,	and	may	

have	a	negative	effect	on	the	health	of	

individuals	residing	in	the	study	area.		By	

increasing	the	emissions	of	greenhouse	

gases,	this	option	has	the	potential	to	

increase	the	negative	effects	of	climate	

change.

Continued	decrease	in	the	amount	of	air	

pollutants,	including	greenhouse	gases,	

emitted	into	the	atmosphere	in	the	study	

area.		By	continuing	to	increase	air	quality,	

this	option	will	positively	affect	the	scenic	

beauty	of	the	area,	as	well	as	the	health	of	

individuals	residing	in	the	study	area.		By	

continuing	to	decrease	the	emissions	of	

greenhouse	gases,	this	option	has	the	

potential	to	decrease	the	negative	effects	of	

climate	change.

The	 results	 of	 the	 air	 emissions	 treatment	 are	 presented	 in	 Table	 10 below.	 We	 present	

them	alongside	 the	results	 for	 the	“Two	Options	Referendum” treatment (see	Section	6	of	

this	 report),	 which	 represent	 the	 most	 appropriate	 basis	 for	 comparison.	 The	 results	

demonstrate	 that	 the	 addition	 of	 air	 emissions	 considerations	 substantially	 decreases	

support	 for	 changing	 GCD operations.	 Specifically,	 only	 about	 37 percent of	 respondents	
HYPOTHETICAL AIR EMISSIONS TREATMENT PAGE | 46

would	 vote	 to	 change	 dam	 operations,	 a	 significant	 decline	 from	 the	 68 percent who	

supported	 changing	 operations	 in	 the	“Two	Options	 Referendum” treatment.	 In	 contrast,	

over	 half	 of	 respondents	 in	 the	 air	 emissions	 treatment	 would	 vote	 to	 maintain	

operations—compared	 to only	 about	 21	 percent	 expressing	 such	 preferences	 in	 the	

baseline	 treatment—and	 over	 one-quarter	 of	 all	 respondents	 would	 pay	 at	 least	 $25	 to	

maintain	 operations.	These	 results	make	clear	 that	 the	non-market	effects	 of	 hydropower	

on	 air	 emissions	 is	 a	 value	 dimension	 highly	 relevant	 to	 any	 debate	 over	 changing	

hydropower	capacity.

Table	10:	Hypothetical	Air	Emissions	Treatment	Results

Hypothetical	Air	

Emissions	Treatment Two	Options	Referendum

Vote	to	change	operations 36.7% 67.8%

		Would	pay	$25 22.4% 46.9%

Vote	to	maintain	operations 51.0% 21.3%

		Would	pay	$25 27.8% 6.9%

Would	not	vote 12.3% 11.0%

We	 should	 note	 that,	 in	 application	 to	 a	 non-hypothetical	 case,	 the	 air	 emissions	 value	

dimension	 should	 be	 included	 along	 with the	 other	 important	 value	 dimensions	 as	

described	 in	 Section	 7.	 More	 generally,	 a	 complete	 account	 of	 non-market	 values	 for	

alternative	dam	operations	 for	any	operational	scenario	will	require	 the	inclusion	of	all	of	

the	significant	and	relevant value	dimensions	applicable	to	that	scenario.
VALIDITY ASSESSMENTS OF STUDY RESULTS PAGE | 47

9. Validity Assessments of Study Results

Prior	 sections	 of	 this	 report	 have	 focused	 on	 the	 effects	 of	 the	 specific	 study	 treatments.	

Here	we	turn	to	the	overall	pattern	of	expressed	preferences	for	GCD	operations,	in	order	to	

evaluate	 the	possible	 sources	 of	 those	preferences.	 In	a	 study	 of	 this	kind,	 the	 “causes”	 of	

preferences	 cannot	 be	 directly	 discerned,	 so	 instead	 we	 will	 focus	 on	 the	 pattern	 of	

correlations	 among	 the	 respondents’	 attributes,	 beliefs,	 and	 values	 and	 their	 stated	

preferences.

Our	analysis	 focuses	on	 the	contribution	of	an	array	of	respondent	characteristics	and	 the	

expressed	 preference	 for	 changing,	 or	 not	 changing,	 dam	 operations.	 Preferences	 are	

captured	in	two	distinct	ways:

1. Continue	 Ops: Prefer	 to	 retain	 current dam operations	 when	 the	 cost	 to	 the	

respondent	is	zero;

2. WTP	to	Cont.	Ops:	Prefer	to	retain	current dam	operations	even	if	the	cost	is	$25	per	

year	in	increased	taxes (would	vote	“yes” on	advisory	referendum)

In	each	case,	those	who	express	the	preference	are	compared	to	all	other	respondents	who	

expressed	a	different	preference	or	who	declined	 to	 vote.11 Given	 that	 these	measures	are	

binary	(i.e.,	coded	as	“yes=1”	and	“no=0”),	we	use	a	logistic	regression	model	to	evaluate	the	

correlates	of	each	of	our	explanatory	variables.

Each	 model	 includes	 as	 explanatory	 variables	 the	 category	 measures	 for	 each	 of	 the	

experimental	 treatments	 used	in	 the	 study.	Thus	 the	effect	 of	each	 of	 the	 treatments,	and	

the	 statistical	 significance	 of	 the	 differences	 in	 each	 treatment	 from	 the	 base	 replication	

case	(Treatment	1),	are	shown.	The	model	includes	the	following	explanatory	variables:

1. Experimental	treatments (#2	 through	#7),	as	defined in	Section	 3: The	 specific	

content	of	each	treatment	is	shown	in	full	in	Appendix	C;

2. Visit	 GC	 in	 the	 Future:	 Using	 question	 W15_futvisitgc	 from	 the	 survey	 (see	

Appendix B	 for	 full	 question	 wording),	 respondents	 were	 asked	 to	 indicate	

																																																							 11 All	models	employ	the	survey	data	weighted	on	gender,	region	of	residence,	age,	and	race/ethnicity	

to	match	the	national	census.
VALIDITY ASSESSMENTS OF STUDY RESULTS PAGE | 48

whether	they	planned to	visit	the	Grand	Canyon	in	the	future	or	not.	Responses	

were	coded	as	1	for	those	that	said	yes,	and	0	for	those	that	said	no.	

3. Policy	 Decisions:	 Based	 on	 the	 question	 W18_dona (see	 Appendix	 B), this	

variable	 measures	 whether	 respondents think	 relying	 on	 measures	 of	 public	

opinion	 on	 environmental	 policy	 choices	 is	 “a	 good	 way” for	 government	

officials	to	make	policy	decisions.	Responses	were	coded	as	1	for	those	that	said	

yes,	and	0	for	those	that	said	no.	

4. Environmentalism	 Index:	 Respondents’	 beliefs	 about	 the	 state	 of	 the

environment,	 using	 an index	 comprised	 of	 seven	 different	 questions	 from	 the	

survey	 (W24_EP1,	 W25_EP2,	 W26_EP3,	 W27_EP4,	 W28_EP5,	 W29_EP10	 and	

W30_EP11).	 These	 questions	 asked	 respondents	 to	 rate	 their level	 of	

environmental	 concern,	 perceptions	 of	 the	 extent	 of	 environmental	 damage,	

views	on	the	robustness	of the	environment,	and	beliefs	about	the	resilience	of	

the	environment to	human	activities.	The	scale	ranges	from	1	to	5,	with	a	value	

of	1	indicating	a	low	level	 of	concern	and	worry	about	 the	environment and	5	

indicating	a	high	level	of	worry	and	concern.

12

5. Income:	 Respondents’	 estimates	 of their annual	 household	 in	 2013	 (based	 on	

survey	questions	W51_inc50	to	W54_inc200).

6. Education:	Respondents’ highest	level	of	educational	achievement, ranging	from	

elementary	or	some	high	school	to	a	doctorate	degree	(from	question	W2_edu).

7. Rural	 (vs.	 Urban	 and	 Suburban):	 A dichotomous	 measure	 of	 whether	

respondents	 live	 in	 rural	 areas	 or	 urban	 and	 suburban	 areas.	 Residence	 in	 a	

rural area	was coded	 as	 1, and	 residence	 in	 urban	 or	 suburban	 locations was

assigned	a	value	of	0	(from	question	W6_rural).

8. Hydropower	 Preference:	 Respondents’	 views	 on	 whether	 hydropower	 is	 a	

preferred way	to	generate	electricity.	Respondents	were	assigned	a	value	of	1	if	

they	viewed hydropower	as	a	preferable	 source	 of	electricity	and	0	if	 they	did	

not (from	question	W33_hydro).

9. Political	 Ideology:	 Respondents’	 self-placement	 on	 a	 scale	 of	 political	 ideology	

ranging	 from	 strongly	 liberal	 (=1)	 to	 strongly	 conservative (=7),	 based	 on	

question	W44_ideol.	

The logistic	regression model	results	are	shown	in	Table	11.	The	values	in	the	columns	are	

estimated	odds	ratios,	where	a	value	greater	than	one	indicates	that	that	there	is	a	positive

relationship	 between the	 explanatory	 variable	 and the	 odds	 of	 respondents	 expressing	 a

preference	or	willingness	to	pay	$25	in	higher	taxes to	maintain dam	operations (models	1	

and	 2,	 respectively),	 while	 a	 value	 below	 one	 indicates	 a	 negative relationship.	 The	

estimated	odds	ratios	that	are	statistically	significant	are	so	indicated	by	the asterisks.

																																																							 12 The	Cronbach’s	alpha	for	the	index	is	0.73.
VALIDITY ASSESSMENTS OF STUDY RESULTS PAGE | 49

Table	11:	Sources	of	Operational	Preferences	and	WTP

Model	1:

Continue	

Operations

at	No	Cost

Model	2:

WTP	$25	to	

Continue

Operations

Group	2	(vs.	Group	1) 0.961 0.307

Group	3	(vs.	Group	1) 1.375 2.773+

Group	4	(vs. Group	1) 3.234*** 7.117***

Group	5	(vs.	Group	1) 2.880*** 7.199***

Group	6	(vs.	Group	1) 5.696*** 13.781***

Group	7	(vs.	Group	1) 6.502*** 15.255***

Visit	GC	in	Future 1.045 1.499*

Policy	Decisions 0.788+ 1.507*

Environ.	Index 0.689*** 0.782*

Income 1.027+ 1.017

Education 0.893* 0.977

Rural	(vs.	Urb	&	Sub) 1.116 1.317

Hydropower	Pref. 1.417** 1.390*

Political	Ideology 1.155*** 1.059

Intercept 0.516 0.022***

Likelihood-ratio 348.825 285.431

Log-Likelihood 2655.330 1653.750

Number	of	Cases 2329 2331

Note:	***	p <	0.001;	**	p <	0.01;	*	p <	0.025;	+ p<0.05	(for	directional,	1-tailed	hypothesis	tests)

As	 indicated	 for respondents	 from	 Group 2 (which	 had	 the	 information	 presented	 in	

randomized	 order), expressed	 preferences	 and	 WTP	 $25 were	 not	 statistically	 different	

from	 the	 replication	 case	 (Group	 1).	However	 for	 Group	 3,	 the	 difference	in	WTP	 $25	for	

maintaining	 current	 dam	 operation	 is	 statistically	 significant (in	 a	 one-tailed	 hypothesis	

test).	 When	 respondents	 could	 choose	 between	 the	 two	 options,	 they were	 significantly	

more	 likely	 to	 express WTP	 $25	 to	 maintain	 dam	 operations	 than	 were	 those	 in	 the	

replication	treatment	(Group	1)	– who	could	vote	against	the	proposal	but	could	not	vote	to	

pay	to	maintain	current	operations.

13

																																																							 13 Another	way	to	understand	this	finding	is	to	see	that,	when	given	the	option	to	choose	to	maintain	

current	dam	operations	and to	pay	$25	in	 taxes	 to	do	so,	 the	 fraction	of	respondents	choosing	 that	

option	is	significantly	greater	than	zero.	This	is	consistent	with	the	findings	described	in	Section	3.
VALIDITY ASSESSMENTS OF STUDY RESULTS PAGE | 50

Group	 4 respondents,	 for	 whom potential	 effects	 of	 changing	 dam	 operations	 on	 Native	

American	businesses	and	communities	were	described,	were	estimated	to	have higher odds	

of	voting	to	retain	current	operations	(Model	1)	and	increased	odds	of	expressing WTP $25	

per	year	in	additional	taxes	to	retain	current	operations (Model	2) in	comparison	to	those	

receiving	the	replication	treatment	(Group	1).	Similar	effects	are seen	for	respondents	from	

Group	 5,	 who	 saw information	 about	 potential	 effects	 on	 rural ranching	 and	 farming	

communities.	For	respondents	in	Group	6, the	cumulative	effect	of	seeing	information	about	

both	Native	American	and	rural	ranching/farming	communities	was	greater	still.

For	 the	 final	experimental	 treatment,	when	respondents	were	exposed	to the	hypothetical	

case	in	which	changing	dam	operations	resulted	in	increased	emissions	(Group	7),	the	odds	

of	 voting	 to	maintain	 current	 operations,	 or	 to	 pay	 $25	in	increased	 taxes	 to	 do	 so,	 were	

increased	 significantly	 relative	 to	 the	 replication	 case	 (Group	 1).	 All	 of	 these	 results	 are	

consistent	 with	 the	treatment-by-treatment findings	 described	in	 Sections	 3 through	8	 of	

this	report.

Respondent	characteristics	also	proved	 to	be	important	predictors	of	preferences	 for	dam	

operations.	These	can	be	summarized	as	follows:

1. Respondents	 who	 indicated	 that	 they	 expected	 to	 visit	 the	 Grand	 Canyon	 in	 the	

future	(Visit	GC	in	the	Future) were	more	likely	to	be	willing	to	pay	$25	to	maintain	

current	dam	operations – relative	 to	 those	who	did	not	plan	 to	visit	 the	Canyon	in	

the	future.	Put	differently,	those	who	did	not	plan	to	visit	the	Canyon	were	less	likely	

to	be	willing	to	pay	$25	in	taxes to	maintain	current operations.

2. Respondents	who	thought	it	was	appropriate	that	policy	makers	rely	on	information	

about	 public	 preferences	 in	 making	 environmental	 policy	 decisions	 (Policy	

Decisions)	were	less	likely	to	vote	to	maintain	current	dam	operations	(Model	1),	but	

were	more	likely	to	be	willing	to	pay	$25	more	in	annual	taxes	to	maintain current

operations	 (Model	 2).	 Put differently,	 those	 who	 did	 not believe	 policy	 makers	

should	 rely	 on	 public	 preference	information	 were	less	likely	 to	 be	 willing	 to	 pay	

more	in	taxes	to	maintain	current	dam operations.

3. Respondents who	 scored	 higher	 on	 the	 environmentalism	 index	 (Environ.	 Index)	

were	significantly	less	likely	to	vote	to	maintain	current	dam	operations	(Model	1)	
VALIDITY ASSESSMENTS OF STUDY RESULTS PAGE | 51

and	to	be	willing	to	pay	$25	more	in	taxes	to	do	so	(Model	2)	than	respondents	who	

scored	lower	on	the	environmentalism	index.

4. Higher	Income had	the	effect	of	slightly	increasing	the	odds	that	respondents	would	

vote	 to	maintain	 current	 dam	 operations	 at	 zero	 cost	 (Model	 1).	However,	 higher	

income	respondents	were	no	more	likely	to	vote	to	increase	taxes	by	$25 per	year	to	

maintain	current	dam	operations	(Model	2).	

5. Higher	 Education	 levels	 had	 the	 effect	 of	 significantly	 decreasing	 the	 odds	 that	

respondents	would	vote	to	maintain	current	dam	operations	(Model	1),	but	had	no	

effect	on	the	expressed	willingness	to	pay	$25	to	do	so	(Model	2).	

6. The	location	 of	 respondents’	 residences	in	 rural,	 suburban	 or	 urban	 areas	 had	 no	

statistically	significant	effect	on	preferences	or	WTP	$25	for	dam	operations.

7. Perceptions	of	hydropower	(Hydropower	Prefs)	as	a	preferable	source	of	electricity	

significantly	 increased	 the	 odds	 of	 preferring	 to	 maintain	 current	 dam	 operations	

(Model	 1),	 and	 of	 expressed	 willingness	 to	 pay	 $25	 in	 additional	 taxes	 to	 do	 so	

(Model	2).

8. Respondents	 who	 described	 themselves	 as	 more	 politically	 conservative	 (Political	

Ideology) were	more	likely	 to	 vote	 to	maintain	 current	 dam	 operations	 (Model	 1),	

but	 were	 no	 more	 likely	 to	 be	 willing	 to	 pay	 $25	 to	 do	 so	 (Model	 2)	 than	

respondents	who	described	themselves	as	less	politically	conservative.

Overall,	then,	the	expressed	preferences	for	dam	operations, and	WTP	$25	to	achieve	those	

options, were	reasonably	and	consistently	related	to	both	the	experimental	treatments	and	

to	 respondent	 attributes and	 beliefs.	 Interpretation	 of	 these	 relationships,	 and	 the	

articulation	of	clear	theoretical	expectations	about	them,	is	complicated	by	the	existence	of	

multiple	 and	 sometimes	 conflicting	 value	 considerations	 that	 survey	 respondents	 must	

engage	 when	 making	 choices	 about	 dam	 operations.	 Nevertheless,	 when	 addressing	 the	

implications	 of	 change	 in	 major	 components	 of	 complex	 coupled	 human	 and	 natural	

systems,	 as	 represented	 by	 the	 dams	 on	 the	 Colorado	 River,	 analysts	 must	 include	 and	

account	for	these	kinds	of	complex	choices.	

We	presented	respondents	with	a	fairly	large	amount	of	information	throughout	the	course	

of	 each	 treatment, and	 it	 is	 possible	 that	 this	 information	 provision	 could	 have	 affected

respondents’	ability	 to	provide	 valid	information in	 two	primary	ways. First,	it	is	possible

that	providing	large	amounts	 of	information	would	 result	in	 responses	 that	do	not	 reflect	
VALIDITY ASSESSMENTS OF STUDY RESULTS PAGE | 52

respondents’	true	preferences	and	values—there	would	be	large	amounts	of	measurement	

error.		This	would	have	the	effect	of	biasing	contrasts	across	treatments	toward	zero.		As	we	

found	statistically	significant	and	substantively	large	differences	across	 treatments,	we	do	

not	believe	general	information	overload	to	be	an	issue.		

Second, and	 more	 worrying	 from	 an	 internal	 validity	 perspective, it	 is	 possible that	

respondents	 across	 the	 seven treatments	 were	 differentially	 affected	 by the	 amount	 of	

information	provided.	We	investigated	this	possibility	in	two	ways.	First,	on	the	final	page	of	

the	 survey	we	 included the	 following	text: “Scholars	 have	 learned	 that	 information	 often	

influences	 the	 way	 in	 which	 people	 answer	 survey	 questions.	 With	 this	 in	 mind,	 we	 are	

interested	in	whether	you	are	taking	the	time	to	read	the	text	that	precedes	each	question.	

So,	in	order	 to	demonstrate	 that	you	have	read	 this	 text,	please	ignore	 the	question	below	

and	click	on	the	blue	dot.”	The	question	referenced	in	the	quote	above	asks	respondents	to	

select	 the	 type	 of	 device	 on	which	 they	 took	 the	 survey.	 Instead	 of	 doing	 so,	 respondents	

should	click	a	blue	dot	on	 the	page.	Table	12 presents	 the	percentage	of	respondents	 that	

clicked	on	the	blue	dot	across	each	of	the	seven	treatments.	

Table	12:	Analysis	of	Attention	and	Effort

Treatment

Percent that 

clicked on blue 

dot

Mean number of 

minutes taken to 

complete survey

1 31.9% 24.8

2 35.9% 27.0

3 37.5% 26.4

4 37.1% 26.8

5 35.3% 26.2

6 44.7% 27.5

7 41.9% 26.6

As	shown	in	Table	12,	 there	is	modest	 variation	in	 the	percentages	of	respondents	across	

treatments	 that	 correctly	 responded	 to	 the	 instructions	 in	 the	 blue	 dot	 question.	 Formal	

statistical	 tests	 indicate	 no	 significant	 differences	 in	 the	 percentage	 of	 respondents	 that	

clicked the	 blue	 dot	 across	 each	 of	 the	 seven	 treatments,	 indicating	 that	 variations	 in	

information	 load	 did	 not	 differentially	 dispose	 respondents	 to	 read	 (or	 ignore)	 the	

information	and	instructions	provided	in	the	survey.	Second,	as	shown	in	the	third	column	

of	Table	12,	we	analyzed	the	mean	number	of	minutes taken	to	complete	the	survey	across	
VALIDITY ASSESSMENTS OF STUDY RESULTS PAGE | 53

each	of	the	seven	treatments, and	though	there	were	nominal	differences	in	time	taken	(on	

average,	treatment	6	took	2.7	minutes	longer	to	complete	than	did	treatment	1),	we found

no	 statistically	 significant	 differences	 across	 the	 treatments.	 Together,	 these	 results	 allay	

our	concerns	about potential bias	resulting from	differential	levels	of	information	provision.	

The	 next	 section	 summarizes	 our	 findings	 and	 discusses	the	 implications	 for	 analyses	 of	

WTP	for	modifying	GCD	operations.
SUMMARY AND CONCLUSIONS PAGE | 54

10. Summary and Conclusions

As	 described	 in	 Section	 2,	 the	 Anthropocene	 epoch	 is	 characterized	 by	 extensive	

interactions	 among natural	 systems	 and	 managed	 human	 institutional	 arrangements.	

Complex	 coupled	 human	 and	 natural	 systems,	 such	 as	 river	 systems	 with	managed	 flows	

intended	 to	 produce	 a	 mix	 of	 ecosystem,	 energy,	 water	 and	 recreational	 resources,	 will	

necessarily	 implicate multiple	 dimensions	 of	 societal	 value.	 Once	 a	 set	 of	 institutional	

arrangements	 are	 in	 place,	 individuals	 and	 communities	 will	 adapt	 (or	 optimize)	 to	 the	

conditions	created	by	those	arrangements.	When	the	larger	society	places	non-market	value	

on	the	preservation	of	those	communities	and	the	ways	of	life	of	their	residents,	changes	in	

system	operations	will	impose	social	costs.

This	study	began	as an	effort	to	address	the	puzzle	that prior	studies	have	found a	positive,	

and	substantial,	willingness	to	pay	to	constrain	production	of hydropower	despite	evidence	

that a	sizable majority	of	the	public	views	hydropower	in	a	positive	light.	After	setting	out	

some	 background	 context	 for increasing	 awareness	 of	 the	 multiple	 dimensions	 of	 nonmarket	values	for	large	coupled	human	and	natural	systems,	we	addressed this	puzzle	from	

several	 directions.	 Specifically,	 in	 the	 preceding	 sections	 of	 this	 report	 we analyzed the	

effects	 of	 variations	 in	 the	 presentation	 of	 information,	 the	 approach	 used	 to	 structure	

choices,	and	the	inclusion	of	additional	relevant	dimensions	of	value on	expressions	of	value.

These	lines	of	inquiry	led	to several	important	results.	

First,	 we	 found	 that	 the	 traditional	 presentation	 of	 the	 choice	 to	 survey	 respondents	 in	

contingent	 valuation	 exercises	 can	 bias	 expression	 of	 value.	 When	 the	 survey	 choice	

architecture	provided	respondents	with	no	way	to	register	WTP	to	avoid	the	option	(as	in	

the	replication	case),	the	results	were	statistically	significantly	skewed	in	favor	of	net	WTP	

to	change	dam	operations.	Our	analysis	suggests that,	by structuring	alternatives	as	a	single	

proposal	 for	 changing	 dam	 operations,	 prior	 studies	 may	 have	 over-estimated	 WTP	 for	

changing	 dam	 operations.	 Our	 alternative	 approach	 frames	 the	 choice	 as	 an	 alternative	

between	two	options,	allowing respondents	to	express	a	positive	value for	maintaining	dam	

operations.	Even	when	we	replicated	the	information	and	considerations	used	in	the	Welsh	
SUMMARY AND CONCLUSIONS PAGE | 55

et	al. (1995)	study,	a	significant	percentage	of	our	respondents	expressed	a	willingness	 to	

pay	to	keep	dam	operations	unchanged.	These	kinds	of	“negative	values”	for	changing	dam	

operations	had	not	been	accounted	for	in	prior	estimations	of	WTP.	

Second,	 our	 analysis	 makes clear	 that	 the	 inclusion	 of	 several	 previously	 omitted

dimensions	of	social	value—including	effects	of	changing	(or	not	changing)	dam	operations	

on Native	 American	 tribes,	 rural	 Western	 communities,	 and	 hypothetical	 levels	 of	 air	

emissions—is	 important	 for	 fully	 capturing	 non-market	 values.	 We	 showed	 that	 these	

additional	 dimensions	 of	 value	 are relevant	 to	 public	 non-market	 value	 expressions

regarding	 dam	 operations,	 and	 thus	 to	 expected	 estimates	 of	 WTP	 for	 the	 effects	 of	

changing	 operations.	 Relative	 to	 a	 baseline	 case	 where	 they	 are	 excluded,	 including	

information	 about	 the	 potential effects	 of	 changing	 dam	 operations	 on	 Native	 American	

tribes,	rural	Western	communities,	and	air	quality significantly	and	substantially decreases	

WTP for	changing	GCD operations.	In	fact,	the	results	suggest	that	there	may	be	a	significant	

fraction	of	 the	population	with	a	positive WTP	 for	maintaining	dam	operations,	a	result	at	

odds	 with	 previous	 research	 on	 the	 issue,	 but	 fully	 consistent	 with	 the	 public’s	

overwhelmingly	positive	view	of	hydropower.

Models	 of	 the	 relationships	 between	 individual	 attributes	 and	 beliefs	 shown	 in	 Section	 9	

demonstrate	 that	 the	 expressions	 of	 value	 obtained	 from	 survey	 respondents	 are	

reasonable	and	plausible,	and	are	consistent	with	an	account	of	choice	in	which	individuals	

are	 forced	 to	 trade	 off	 important	 dimensions	 of	 value.	 In	 short,	 the	 findings	 provide	 a	

reasonable	 and	 theoretically	 consistent	 account	 of	 non-use	 value	 expression	 for	 nonrejectable	goods	when	changes	in	complex	institutional	arrangements	are	being	considered.

In	 sum,	 when	 accounting	 for	 the	 broad	 range	 of	 social	 values that	 are	 affected	 by	 the	

management	 of	 the	 GCD,	 our	 pilot	 study	 demonstrates	 that	 public	 willingness	 to	 pay	 for	

changes	in	dam	operations	is	more	complex	than	has	been portrayed	in	earlier	CV	studies.	

One	implication	is	that	prior	measures	of	non-market	and	non-use	values	for	changes	in	the	

operation	of	the	GCD	may	have	substantially	over-estimated	public	WTP	for	such	changes.	A	

broader	implication	is	that	the	characterization	and	measurement	of	non-market	values	for	

hydropower	 operations,	 and	 for	 the	 operation	 of	 the	 systems	 of	 generators	 on	 dams	 in	

major	U.S. rivers,	will	require	an	approach	that	recognizes	that	these	operations	constitute	
SUMMARY AND CONCLUSIONS PAGE | 56

sets	 of	 institutional	 arrangements	 to	 which	 communities	 and	 individuals	 have	 adapted.	

Changes	 in	 these	 institutional	 arrangements	 may	 disrupt	 communities,	 their	 ways	 of	 life,	

and	may	 therefore	 engage	 societal	 non-market	 values	in	ways	 not	 captured	in	 traditional	

studies.	Thus,	if	policy	makers	are	to	take	into	account	fairness	and	the	social	externalities	

associated	with	those	communities	that	have	adapted	to	existing	institutional	arrangements	

of	management	of	modern	rivers,	a	more	balanced	and	inclusive	approach	to	measurement	

of	non-market	values	is	required.
APPENDIX A: SURVEY DATA COLLECTION PAGE | 57

Appendix A: Survey Data Collection

This	study	is	based	on	a	nationwide	 Internet	sample	of	2,465	respondents	 from	 the	48	

contiguous	states	conducted	April	22-24,	2014.

Background. Technological	 developments	 and	 telecommunications	 trends,	 such	 as	 the	

declining	 number	 of	 wired	 phones,	 the	 increasing	 use	 of	 cellular	 phones,	 and	 the	

continuing	 expansion	 of	 high	 speed	 Internet	 services,	 have	 made	 probabilistic	 (often	

referred	to	as	“random”)	sampling	of	the	U.S. national	population	for	the	administration	

of	lengthy	surveys	on	complex	issues	infeasible	for	several	reasons.

o The	 total	 universe	 of	 households	 without	 phone	 service	 of	 any	 kind	 is	

unknown.	

o The	 wired	telephone	 system no	 longer	 includes	 a	 sufficient	 number	 of	 U.S.

households	to	represent	the	national	population,	and	members	of	households	

that	 do	 have	 wired	 service	 differ	 systematically	 from	 households	 without	

wired	phones.	

o The	 number	 of	 households	 with	 wired	 phones	 that	 are	 used	 almost	

exclusively	 for	 other	 than	 routine	 phone	 calls,	 such	 as	 home	 alarms	 or	

medical	alert	services,	is	unknown.	

o The	numbers	of	individuals	and	households	having	both	a	wired	phone	and	a	

cell	phone	or	those	having	more	than	one	cell	phone	are	unknown.	

o The	number	of	households	and	individuals	having	access	to	 Internet	service	

suitable	for	taking	Internet	surveys	is	unknown.	

o The	numbers	of	individuals	who	have	access	 to	 Internet	services	 from	 their	

place	 of	 employment	 is	 unknown,	 and	 of	 those,	 the	 number of	 individuals	

who	are	allowed	to	take	surveys	while	at	work	is	unknown.	

o Cellular	 telephones	 are	 unsuitable	 for	 lengthy	 surveys	 that	 may	 distract	

respondents	who	are	 otherwise	 occupied,	and	 surveys	 conducted	 using	 cell	

phones	 may	 incur	 costs	 to	 prospective	 respondents	 that	 discourage	 survey	

participation.	

o Face-to-face	 interviews	 or	 printed	 postal	 surveys	 of	 the	 U.S. general	 public	

require	 long	 collection	 periods	 and	 are	 prohibitively	 expensive	 for	 many	

research	projects.	

Increasingly,	 academic-quality	 opinion	 survey	 research	 of	 the	 U.S. public	 on	 complex	

subjects,	such	as	privacy	and	security,	is being	conducted	via	the	Internet.	These	factors	

present	 special	 challenges	 for	 probabilistic	 sampling	 because	 insufficient	 information	
APPENDIX A: SURVEY DATA COLLECTION PAGE | 58

exists	about	rapidly	evolving	telecommunication	patterns,	Internet	accessibility,	and	the	

demographic	composition	of	those	who have	suitable	Internet	access.

With	increasing	Internet	access,	the	demographics	of	the	online	population	are	becoming	

more	 representative	 of	 the	 U.S. population	 as	 a	 whole,	 but	 samples	 recruited	 to	

participate	 in	 Internet	 surveys	 cannot	 be	 truly	 random	 samples	 of	 the	 U.S. public.	 All	

surveys,	 regardless	 of	 collection	 methods,	 include	 an	 element	 of	 self-selection	 because	

even	if	a	perfectly	random	sample	could	be	constructed,	the	final	decision	to	participate	

must	be	made	voluntarily	by	each	respondent,	and	thus	some	degree	of	self-selection	is	

unavoidable.	This	means	 that	even	when	derived	 from	a	 theoretically	perfectly	random	

sample, the	demographic	characteristics	of	survey	respondents	may	not	perfectly	reflect	

U.S. population	parameters.	Non-probabilistic	samples,	such	as	those	used	to	administer	

surveys	 of	 the	 general	 public	 via	 the	 Internet,	involve	 greater	 degrees	 of	 self-selection	

because	participants	first	voluntarily	agree	to	enter	a	pool	or	stream	of	citizens	willing	to	

take	surveys	online,	and	then	each	member	of	that	group	who	attempts	to	take	an	online	

survey	 must	 decide	 whether	 or	 not	 to	 participate	 in	 a	 given	 survey	 opportunity.	 This	

requires the	 administration	 of	 Internet	 surveys	 that	 are	 as	 demographically	

representative	 as	 possible,	 careful	 weighting	 of	 the	 sample	 to	 match	 U.S. Census	

characteristics,	 and	 it	 warrants	 caution	 in	 presenting	 findings	 as	 statistically	

representative	of	views	of the	entire	adult	U.S. population.

Sampling. Sampling	for	this	survey	was	accomplished	by	Survey	Sampling	International	

(SSI)	using	a	proprietary	methodology	known	as	Dynamix,	which	provides	direct	access	

to	 millions	 of	 potential	 survey	 respondents,	 including	 members	 of	 34	 standing	 panels	

plus	a	 variety	 of	 online	 communities,	 social	media,	and	affiliate	partnerships	 to	 build	a	

constantly	 evolving	 stream	 of	 potential	 survey	 participants.	 Prospective	 survey	

respondents	 access	SSI’s	website	 directly	 or	 through	 trusted	 partnerships	 or	 via	 social	

media	 or	 numerous	 other	 Internet	 portals	 where	 invitations	 are	 posted	 to	 maximize	

diversity.	 Each	 individual	 is	 screened	 and	 dynamically	 profiled.	 Rather	 than	 being	

sampled	for	specific	individual	surveys,	respondents	are	offered	one	of	various	available	

projects	 for	which	 they	 qualify	 at	 the	 time	 they	 choose	 to	 participate	in	 a	 survey.	 This	

increases	 the	 probability	 of	 survey	 engagement,	 reduces	 dropout	 rates,	 and	 allows	

prospects	to	take	a	survey	at	a	time	of	their	convenience.
APPENDIX A: SURVEY DATA COLLECTION PAGE | 59

If	 certain	 demographic	 categories	 are	 underrepresented	 during	 the	 dynamic	 sampling	

process,	 email	 invitations	 are	 employed	 to	 bring	 the	 sample	 into	 balance	 with	 key	

population	parameters.	Assuming	a	sample	size	of	1,000,	the	sample	frame	and	selection	

process	would	achieve	comparable	results	within	a	+	3	percent	range	19	times	out	of	20.	

Security	checks	and	quality	verifications	are	used	on	all	sources	before	respondents	can	

begin	any	survey.	All	external	sources	undergo	a	partner	verification	process	that	scores	

individuals	 based	 on	 a	 series	 of	 quality	 control	 checkpoints,	 including	 digital	

fingerprinting	to	prevent	duplication,	spot	checking	via	third	party	verification	to	prove	

identity,	 benchmarking	 against	 known	 external	 data	 points,	 and an	 algorithm	 that	

dynamically	 monitors	 the	 sample	 on	 a	 number	 of	 personality	 and	 psychographic	

measures.

A	variety	of	incentives	from	SSI	or	from	affiliate	organizations	are	employed	based	on	the	

nature	 and	 length	 of	 surveys	 and	 progress	 of	 the	 dynamic	 sampling	 process.	 This	

sampling	 process	 broadens	 access	 beyond	 standing	 panel	 memberships	 by	 including	

individuals	who	are	not	interested	in	joining	a	research	panel	and	who	may	only	rarely	

choose	to	participate	in	online	survey	research.	It	does	not	limit	the	sampling	process	to	

one	or	a	few	sample	sources,	one	or	a	few	modes	of	contact, or	a	single	selection	method.

But	 regardless	 of	 its	 advantages,	 neither	 Dynamix	 nor	 any	 other	 Internet	 sampling	

methodology	 provides	a	 probabilistic	 sample	 that	 can	 be	 represented	as	 truly	 random.	

While	 the	 possibilities	 of	 systematic	 bias	 can	 be	 reduced	 to	minimum	levels	 that	allow	

replication	 of	 survey	 findings,	 they	 cannot	 be	 eliminated	 entirely.	 One	 source	 of	

systematic	bias	that	can	be	minimized	is	demographic	representativeness.

Sample	 Weighting. To	 assure	 that	 the	 sample	 matched	 the	 most	 recent	 Census	

estimates	as	 closely	as	 possible,	we	weighted	 the	 data.	The	weighting	 process	involves	

three	 related	 steps.	 The	 first	 step	 is	 to	 calculate	 for	 each	 survey	 respondent	 the	

proportion	of	the	U.S. population	for	the	survey	year	that	shares	the	same	demographic	

characteristics	of	gender,	age,	race,	ethnicity,	and	census	region	as	 the	respondent.	The	

second	 step	 is	 to	 calculate	 the	 proportion	 of	 fellow	 survey	 participants	 who	 share	 the	

same	 demographic	 characteristics	 as	 the	 respondent	 being	 weighted.	 Finally,	 the	
APPENDIX A: SURVEY DATA COLLECTION PAGE | 60

proportion	of	the	general	population	sharing	those	demographic	attributes	is	divided	by	

the	proportion	of	survey	respondents	sharing	those	same	characteristics.	The	result	is	a	

weight	factor	that	can	be	applied	to	responses	from	that	individual	survey	participant	to	

adjust	 them	 to	 general	 population	 characteristics.	 A	 weight	 value	 of	 one	 means	 that	

responses	 from	 a	 specified	 participant	 are	 used	 without	 adjustment.	 A	 weight	 value	

greater	 than	one	means	 that	a	participant	with	a	given	set	of	demographic	attributes	is	

underrepresented	 in	 the	 survey	 sample	 (relative	 to	 the	 general	 population),	 and	

responses	from	that	participant	receive	greater	statistical	emphasis	than	responses	from	

survey	participants	who	are	represented	in	direct	proportion	to	the	general	population.	

Conversely,	a	weight	value	smaller	than	one	means	that	a	respondent	having	a	given	set	

of	 demographic	 attributes	 is	 overrepresented	 in	 the	 survey	 sample	 (relative	 to	 the	

general	 population),	 and	 responses	 from	 that	 participant	 receive	 less	 emphasis	 than	

fellow	respondents	who	are	represented	in	direct	proportion	to	the	general	population.	

We	calculated	weight	factors	to	six	decimal	places.

Demographic	 Representativeness. Table	 A1 compares	 key	 national	 and	 regional	

population	 parameters	 to	 the	 demographic	 weighted	 characteristics	 of	 respondents	 to	

this	 survey.	Notice	 that,	after	weighting,	educational	attainment	was	 higher	among	 our	

respondents	 than	 for	 the	 national	 population	 as	 a	 whole.	 But	 overall,	 the	 weighted	

demographic	attributes	of	respondents	to	this	survey	have	a	high	level	of	comparability	

to	national	population	demographics.
APPENDIX A: SURVEY DATA COLLECTION PAGE | 61

Table	A	1:	Demographic	Representativeness	of	Respondents	(Weighted)

Demographic

Category

%	U.S.

Population	(Ages	

18+)

%

Survey

Gender14

Women 51.4 50.7

Men 48.6 49.3

Age15

18–29 22.0 21.3

30–49 34.6 35.8

50	and	above 43.4 42.8

Education16

High	School	Graduate	or	higher 86.8 98.1

Bachelor’s Degree	or	higher 28.2 47.5

Ethnicity17

Hispanic 14.8 13.8

Non-Hispanic 85.2 86.4

Race18

White	alone 79.3 80.2

Black	or	African	American	alone 12.5 11.4

American	Indian	/	Alaska	Native	alone 1.1 0.3

Asian	alone 5.2 4.8

Native	Hawaiian	or	other	Pacific				

Islander	alone 0.2 >0.1

Two	or	more	races 1.7 2.7

Household	Income19

$0–49,999 48.6 51.8

$50–99,999 29.9 32.9

$100–199,999 17.0 10.1

$200,000	or	more 4.6 5.1

Data	 Collection. For	 the	 protection	 of	 participants,	 both	 the	 survey	 questions and	 the	

survey	 protocol	 were	 approved	 by	 the	 Institutional	 Review	 Board	 of	 the	 University	 of	

Oklahoma.	The	instrument	was	programmed	to	allow	the	survey	to	be	self-administered	

																																																							 14U.S.	Census	Bureau	2013a.

15 Ibid.

16 U.S.	Census	Bureau	2013b. 17 U.S.	Census	Bureau	2013a. 18 Ibid. 19 U.S.	Census	Bureau	2012.
APPENDIX A: SURVEY DATA COLLECTION PAGE | 62

at	the	preferred	time	and	pace	of	each	respondent	within	clearly	defined	time	constraints.	

To	afford	continuity	of	attention	and	to	make	best	use	of	factual	information	provided	to	

each	respondent	during	the	course	of	the	survey,	a	maximum	of	45	minutes	was	allowed	

for	 completion	 of	any	 single	Web	 page	in	 the	 survey	 (typically	 containing	 one	 to	 three	

survey	questions),	and	a	maximum	 total	elapsed	 time	 of	 two	hours	 from	start	 to	 finish	

was	 allowed	 to	 complete	 the	 survey.	 Average	 completion	 time	 was	 27.8	 minutes.	

Participation	 was	 restricted	 only	 to	 individuals	 18	 years	 of	 age	 or	 above.	 Each	

respondent	who	completed	the	survey	received	points	credited	by	SSI	or	incentives	from	

affiliates	equal	to	a	five	dollar	stipend.	Decisions	to	participate	were	entirely	voluntary.	
APPENDIX B: FREQUENCY REPORT AND DATA SUMMARY PAGE | 63

Appendix B: Frequency Report and Data Summary

W1_age:	How	old	are	you?

Mean

WATER SURVEY 2014 46.1

W2_edu:	What	is	the	highest	level	of	education	you	have	COMPLETED?

WATER	SURVEY 2014 %

1-Elementary	or	some	high	school 1.9

2-High	school	graduate/GED 17.8

3-Some	college/vocational	school 32.8

4-College	graduate 28.2

5-Some	graduate	work 4.4

6-Master’s	degree 11.2

7-Doctorate	(of	any	type) 2.4

8-Other	degree	[verbatim] 1.3

W3_gend:	Are	you	male	or	female?

Female Male

% 0 1

WATER SURVEY 2014 50.7 49.3

W6_rural: Which	of	the	following	categories	best	describes	the	location	of	your	primary	residence?

WATER	SURVEY 2014 %

1-Urban 35.5

2-Suburban 47.9

3-Rural 16.6

W6_oldvote:	Think	about	a	situation	in	which	you	had	the	opportunity	to	vote	on	this	proposal.	If	passage	

of	this	proposal	would	not	cost	you	anything	would	you	support	this	proposal?	[Asked	only	of	Groups	1	and	

2]

I	would	choose	not	to

No Yes vote	on	this	proposal

% 1 2 3

Group	1 17.2 70.6 12.3

Group	2 17.2 71.0 11.8
APPENDIX B: FREQUENCY REPORT AND DATA SUMMARY PAGE | 64

W7_newvote:	Think	about	a	situation	in	which	you	had	an	opportunity	to	vote	for	Option	1	or	Option	2.		

Keeping	in	mind	all	of	the	potential	effects	described	for	each	option	above,	and	if	adoption	of	either	option	

would	not	cost	you	anything,	would	you	vote	for	Option	1	or	Option	2?	[Asked	only	of	Groups	3-7]

I	would	choose	not	to

Maintain Change vote	on	either	option

% 1 2 3

Group	3 21.3 67.8 11.0

Group	4 39.2 49.5 11.4

Group	5 37.1 49.1 13.8

Group	6 51.0 36.7 12.3

Group	7 57.1 32.4 10.5

W8_cv:	Would	you	vote	for	this	proposal/option	[Groups	1	and	2	get	“proposal”.	Groups	3-7	get	“option”]	if	

adoption	of	this	option	would	cost	your	household	$_25_	in	increased	taxes	every	year for	the	foreseeable	

future?	[Groups	1	and	2	voted	to	change	dam	operations	or	not]

Definitely Probably Probably Definitely

No No Not	sure Yes Yes

% 1 2 3 4 5

Group	1 14.6 18.1 20.8 33.3 13.2

Group	2 17.3 16.5 23.7 30.2 12.2

[Groups	3-7	voted	for	one of	two	options	– change	dam	operations	or	retain	current	operations]

Definitely Probably Probably Definitely

No No Not	sure Yes Yes

% 1 2 3 4 5

Group	3-7	Respondents	Who	Originally	Voted	to	Maintain	Dam	Operations

Group	3 12.3 22.9 32.3 29.2 3.2

Group	4 9.7 30.2 18.1 36.0 6.1

Group	5 13.8 16.1 25.9 33.4 10.8

Group	6 6.1 13.5 25.8 36.5 18.2

Group	7 12.2 13.1 21.3 47.0 6.5

Definitely Probably Probably Definitely

No No Not	sure Yes Yes

% 1 2 3 4 5

Group	3-7	Respondents	Who	Originally	Voted	to Change	Dam	Operations

Group	3 5.5 12.4 12.9 52.7 16.6

Group	4 4.7 12.9 15.7 44.5 22.3

Group	5 3.2 4.6 23.3 50.3 18.7

Group	6 5.8 7.5 25.6 44.3 16.9

Group	7 2.1 5.8 23.8 57.9 10.4

W11_tax:	Do	you	believe	your	taxes	will	increase	if	this	proposal/option [Groups	1	and	2	get	“proposal”.	

Groups	3-7	get	“option”]	passes?

No Yes

% 0 1

WATER	SURVEY 2014 25.4 74.6
APPENDIX B: FREQUENCY REPORT AND DATA SUMMARY PAGE | 65

W12_consider:	Do	you	think	public	officials	will	consider	the	results	of	this	study,	along	with	other	

evidence,	in	deciding	how	Glen	Canyon	Dam	should	be	operated	in	the	future?

No Yes

% 0 1

WATER	SURVEY 2014 41.0 59.0

W13_should: Do	you	think	that	public	officials	should consider	the	results	of	this	study,	along	with	other	

evidence,	in	deciding	how	Glen	Canyon	Dam	should	be	operated	in	the	future?

No Yes

% 0 1

WATER	SURVEY 2014 5.7 94.3

W14_visirgc: Have	you	ever	been	to	the	Grand	Canyon?

No Yes

% 0 1

WATER	SURVEY 2014 61.0 39.0

W15_futvisitgc: Do	you	plan	to	visit	the	Grand	Canyon	in	the	future?

No Yes

% 0 1

WATER	SURVEY 2014 29.0 71.0

W18_dona: Is	this	a	good	way	for	government	officials	to	make	policy	choices	about	options	to	address	

environmental	problems?

No Yes

% 0 1

WATER	SURVEY 2014 24.6 75.4

W24_EP1:	The	so-called	“ecological	crisis”	facing	humankind	has	been	greatly	exaggerated.

Strongly	Disagree Strongly	Agree

1 					2 								3 									4 											5

WATER	

SURVEY 2014

28.1 24.8 24.6 13.4 9.1

W25_EP2:	Humans	will	eventually	learn	enough	about	how	nature	works	to	be	able	to	control	it.

Strongly	Disagree Strongly	Agree

1 					2 								3 								4 												5

WATER	

SURVEY 2014

21.6 26.1 30.9 14.7 6.8
APPENDIX B: FREQUENCY REPORT AND DATA SUMMARY PAGE | 66

W26_EP3:	Humans	are	severely	abusing	the	environment.

Strongly	Disagree Strongly	Agree

1 					2 								3 									4 											5

WATER	

SURVEY 2014 3.7 5.5 18.0 31.3 41.4

W27_EP4: Humans	were	meant	to	rule	over	the	rest	of	nature.

Strongly	Disagree Strongly	Agree

1 						2 								3 								4 												5

WATER	

SURVEY 2014 32.8 20.8 25.7 11.7 9.0

W28_EP5:	When	humans	interfere	with	nature	it	often	produces	disastrous	consequences.

Strongly	Disagree Strongly	Agree

1 					2 								3 									4 												5

WATER	

SURVEY 2014 2.4 5.3 22.4 35.5 34.5

W29_EP10:	We	are	approaching	the	limit	of	the	number	of	people	the	earth	can	support.

Strongly	Disagree Strongly	Agree

1 					2 								3 									4 												5

WATER	

SURVEY 2014 11.9 14.1 29.9 25.8 18.4

W30_EP11: The	balance	of	nature	is	strong	enough	to	cope	with	the	impacts	of	modern	industrial	nations.

Strongly	Disagree Strongly	Agree

1 					2 								3 									4 												5

WATER	

SURVEY 2014 24.7 31.2 26.5 12.1 5.4

W33_hydro:	Electricity	from	hydropower	using	dams

Preferable Not	Preferable

% 9 					10 										NA

W33_hydro_prf 56.9 9.3 33.8

W44_ideol:	On	a	scale	of	political	ideology,	individuals	can	be	arranged	from	strongly	liberal	to	strongly	

conservative.	Which	of	the	following	categories	best	describes	your	views?

Strongly	liberal Middle	of	the	road Strongly	conservative

% 		1 			2 			3 							4 							5 									6 										7

WATER	

SURVEY 2014 7.4 14.4 11.8 34.3 12.6 13.7 5.9
APPENDIX B: FREQUENCY REPORT AND DATA SUMMARY PAGE | 67

W45_Hisp: Do	you	consider	yourself	to	be	Hispanic,	Latino,	or	Spanish	or	to	have	Hispanic,	Latino,	or	

Spanish	origins?

			No 										Yes

% 			0 																																																											1

WATER	SURVEY 2014 86.4 13.6

W46_race: Which	of	the	following	best	describes	your	race?

WATER	SURVEY 2014 %

1-White 80.2

2-Black,	African	American,	or	Negro 11.4

3-American	Indian	or	Alaska Native 0.3

4-Asian 4.8

5-Native	Hawaiian	or	Pacific	Islander 0.0

6-Two	or	more	races 0.8

7-Some	other	race	(please	specify) 2.7
APPENDIX C: EXPERIMENTAL SURVEY TREATMENTS PAGE | 68

Appendix C: Experimental Survey Treatments

Treatment	1- Replication	Treatment
APPENDIX C: EXPERIMENTAL SURVEY TREATMENTS PAGE | 69
APPENDIX C: EXPERIMENTAL SURVEY TREATMENTS PAGE | 70
APPENDIX C: EXPERIMENTAL SURVEY TREATMENTS PAGE | 71
APPENDIX C: EXPERIMENTAL SURVEY TREATMENTS PAGE | 72

Treatment	2- Randomized	Information	Treatment

This	 treatment	 is	 identical	 to	 the	 replication	 treatment,	 but	 information	 blocks	 and	 proposal	

bullets	were	presented	in	randomized	order. Therefore	the	images	are	not	shown	here.

Treatment	3- Two	Options	Treatment.

This	 treatment	 is	 identical	 to	 the	 Randomized	 Information	 Treatment,	 but	 we	 replace	 the	

“Proposal”	image	and	wording with	the	following:

The	 question	 asking	 whether	 the	 respondent	 would	 pay	 $25	 to	 support	 their	 preferred	 option	

took	the	following	form:
APPENDIX C: EXPERIMENTAL SURVEY TREATMENTS PAGE | 73

Treatment	4- Native	American	Treatment

This	 treatment	added the following	information	 block	 to	 the	 page	 describing	 natural	 resources	

and	communities	in	the	study	area:

We	add	 the	 following	information	block	 to	 the	page	describing	concerns	over	natural	resources	

and	communities	in	the	study	area:
APPENDIX C: EXPERIMENTAL SURVEY TREATMENTS PAGE | 74

We	add	the	following	information	block	to	the	page	describing	potential	effects	of	changing	dam	

operations	on	the	natural	resources	and	communities	in	the	study	area:

We	add	the	 following	information	to	the	table, presenting	respondents	with	 the	 two	options	 for	

operating	the	dam:
APPENDIX C: EXPERIMENTAL SURVEY TREATMENTS PAGE | 75

Treatment	5- Rural	Western	Community	Treatment

For	 this	 treatment	 we	 add	 the	 following	 information	 block	 to	 the	 page	 describing	 natural	

resources	and	communities	in	the	study	area:

We	add	 the	 following	information	block	 to	 the	page	describing	concerns	over	natural	resources	

and	communities	in	the	study	area:
APPENDIX C: EXPERIMENTAL SURVEY TREATMENTS PAGE | 76

We	add	the	following	information	block	to	the	page	describing	potential	effects	of	changing	dam	

operations	on	the	natural	resources	and	communities	in	the	study	area:

We	add	the	 following	information	to	the	table, presenting	respondents	with	 the	 two	options	 for	

operating	the	dam:
APPENDIX C: EXPERIMENTAL SURVEY TREATMENTS PAGE | 77

Treatment	6- Native	American	+	Rural	Western	Community	Treatment

For	 this	 treatment	 we	 add	 all	 of	 the	 information	 blocks	 presented	 separately	 in	 the	 Native	

American	and	Rural	Western	Community	treatments. The	components	are	all	shown in	the	prior	

two	subsections,	and	are	therefore	not	repeated	here.

Treatment	7- Hypothetical	Air	Emissions	Treatment

For	 this	 treatment	 we add	 the	 following	 information	 block	 to	 the	 page	 describing	 natural	

resources	and	communities	in	the	study	area:

We	add	 the	 following	information	block	 to	 the	page	describing	concerns	over	natural	resources	

and	communities	in	the	study	area:
APPENDIX C: EXPERIMENTAL SURVEY TREATMENTS PAGE | 78

We	add	the	following	information	block	to	the	page	describing	potential	effects	of	changing	dam	

operations	on	the	natural	resources	and	communities	in	the	study	area:

We	add	the	 following	information	to	the	table, presenting	respondents	with	 the	 two	options	 for	

operating	the	dam:
REFERENCES PAGE | 79

References

Anseel,	F.,	F.	Lievens,	E.	Schollaert,	and	B.	Choragwicka.	2010.	“Response	Rates	in	

Organizational	Science,	1995-2008:	A	Meta-Analytic	Review	and	Guidelines	for	Survey	

Researchers.” Journal	of	Business	Psychology 25(3):335-349.

Atkinson,	G.,	B.	Day,	S.	Mourato,	and	C.	Palmer. 2004.	“Amenity	or	Eyesore:	Negative	

Willingness	to	Pay	for	Options	to	Replace	Electricity	Transmissions	Towers.”	Applied	

Economics	Letters 11(4):203-208.

Berrens,	B.,	A.	Bohara,	H.	Jenkins-Smith,	C.	Silva	and	D.	Weimer.	2003.	“The	Advent	of	

Internet	Surveys	for	Political	Research:	A	Comparison	of	Telephone	and	Internet	

Samples.”	Political	Analysis 11:1-23.

Berrens,	R.,	D.	Brookshire,	P.	Ganderton	and	M.	McKee.	1998.		“Exploring	Nonmarket	Values	

for	the	Social	Impacts	of	Environmental	Policy	Change.”		Resource	and	Energy	

Economics	20(2):117-137.

Bishop	,	R., K. Boyle,	M.	Welsh,	R.	Baumgartner	and	P.	Rathbun.	1987.		Glen	Canyon	Dam	

Releases	and	Downstream	Recreation:	An	Analysis	of	User	Preferences	and	Economic	

Values. Glen	Canyon	Environmental	Studies	Report	No.	27/87.	Madison	WI:	Hagler	

Bailly	Consulting, Inc.	Pp.	188.	Springfield	VA:	National	Technical	Information	Service.	

NTIS	No.	PB88-183546?AS

Bohara,	A.,	J.	Kerkvliet,	and	R.	Berrens.	2001.	“Addressing	Negative	Willingness	to	Pay	in	

Dichotomous	Choice	Contingent	Valuation.”	Environmental	and	Resource	Economics	

20(3):173-195.

Bromley,	D.. 2000.	“Program	Evaluation	and	the	Purpose	of	Rivers.”	Journal	of	Contemporary	

Water	Research	and	Education. 116(1):1-4.

Castle,	E..		2000.	“Reflections	on	Water	Policy	and	Scholarship	in	Economics.”	Journal	of	

Contemporary	Water	Research	and	Education. 116(1):1-7.

Chimeli,	A.,	R.	Boyd	and	D.	Adams.	2012.		“International	Timber	Markets	and	Tropical	

Deforestation:	Evidence	from	Prices.” Applied	Economics 44(10):1303-1313.	

Crutzen,	P.	2002.	“Geology	of	mankind.”	Nature 415:23.

Douglass,	A.,	and	D.	Harpman.	1995.	“Estimating	Recreation	Employment	Effects	with	

IMPLAN	in	the	Glen	Canyon	Dam	Region.”		Journal	of	Environmental	Management

44(3):233-247.

Gillings,	M.,	and	E.	Hagan-Lawson.	2014.		“The	Cost	of	Living	in	the	Anthropocene.”	Earth	

Perspectives 1:2	doi:10.1186/2194-6434-1-2.

Green,	C.,	T	Grijalva	and	S.	Kroll.		“Social	Capital	and	the	Value	of	Hunting	Club	

Memberships.” Human	Dimensions	of	Wildlife 9(1):57-68.

Harpman,	D.,	M.	Welsh	and	R.	Bishop.	1995.		“Nonuse	Economic	Value:	Emerging	Policy	

Analysis	Tool.”	Rivers 4(4):280-291.

Herron,	K.,	H.	Jenkins-Smith,	and	C. Silva.	2012.	US	Public	Perspectives	on	Security.	Sandia	

Report:	SAND2012-0165,	Albuquerque,	NM:	Sandia	National	Laboratories.
REFERENCES PAGE | 80

Jacobs,	J.. 2011.		“The	Sustainability	of	Water	Resources	in	the	Colorado	River	Basin.”		The	

Bridge:	Linking	Engineering	and	Society. 41(4):6-12.

Jenkins-Smith,	H.	and	K.	Herron.	2008. American	Security	Perspectives:	Public	Views	on	

Energy,	Environment,	Nuclear	Weapons	and	Terrorism:	2008.	Sandia	Report:	

SAND2008-8027,	Albuquerque,	NM:	Sandia	National	Laboratories.

Kareiva,	P., M.	Marvier,	and	R.	Lalalz.	2012.	“Conservation	in	the	Anthropocene:	Beyond	

Fragility	and	Solitude.” Breakthrough	Journal. Retrieved	online	from

http://thebreakthrough.org/index.php/journal/past-issues/issue-2/conservation-inthe-anthropocene

Krosnick,	J.	A.,	and	S. Presser.	2010.	Question	and	questionnaire	design.	Handbook	of	survey	

research. 2:263-314.

Liu,	J.,	T.	Dietz,	S.	Carpenter	et	al.. 2007.	“Complexity	of	Coupled	Human	and	Natural	

Systems.”	Science. 317:1513-1517.

Loomis,	J.,	A.	Douglass,	and	D.	Harpman.	2005.		Recreation	Use	Values	and	Nonuse	Values	of

Glen	and	Grand	Canyons.	Pp.	153-164.	In:	S.	Gloss,	J.	Lovich	and	T.	Melis.	Eds.	The	

State	of	the	Colorado	Ecosystem	in	the	Grand	Canyon:	U.S.	Geological	Survey	Circular	

1282,	220	pp.

Loureiro,	M.,	J.	Loomis,	and	L.	Nahuelhual. 2004.		“A	Comparison	of	a	Parametric	and	a	Nonparametric	Method	to	Value	a	Non-rejectable	Public	Good.”		Journal	of	Forest	

Economics. 10(2):61-74.

Lovich,	J.,	and	T.	Melis.	2007.		“The	State	of	the	Colorado	River	Ecosystem	in	the	Grand	

Canyon:	Lessons	from	10	Years	of	Adaptive	Ecosystem	Management.”	Int.	J.	River	

Basin	Management. 5(3):207-217.

U.S.	Census	Bureau,	2012	American	Community	Survey:	Income	in	the	Past	12	Months:	1-

Year	Estimates	in	2012	Inflation	Adjusted	Dollars.	Washington,	DC.	Retrieved	online	

from

http://factfinder2.census.gov/faces/tableservices/jsf/pages/productview.xhtml?pid=

ACS_12_1YR_S1901&prodType=table.	Last	accessed:	February	25,	2014.

U.S.	Census	Bureau.	Population	Division.	2013a.	Annual	Estimates	of	the	Resident	

Population	by	Sex,	Age,	Race,	and	Hispanic	Origin	for	the	United	States	and	States:	

April	1,	2010	to	July	1,	2012.	Washington	DC.	Retrieved	online	from	

http://www.census.gov/popest/data/datasets.html.	Last	accessed:	February	25,	2014.

U.S.	Census	Bureau.	Population	Division.	2013b.	Educational	Attainment	in	the	United	States,	

Table 1:	Educational	Attainment	of	the	Population	18	Years	and	Over.	Washington,	DC.	

Retrieved	online	from

http://www.census.gov/hhes/socdemo/education/data/cps/2013/tables.html.	Last	

accessed:	February	25,	2014.	

Wang,	W.,	D.	Rothschild,	S.	Goel,	and	A.	Gelman.	2014. “Forecasting	Elections	with	NonRepresentative	Polls,”	Preprint	submitted	to	International	Journal	of	Forecasting

March	31,	2014.

Welsh,	M.,	R.	Bishop,	M.	Phillps,	and	R.	Baumgartner.	1995.	GCES	Non-Use	Value	Study:	GCES	

Non-Use	Values	Final	Study	Report. Hagler	Bailly	Consulting, Inc. University	Research	

Park,	Madison	WI.
4/27/2015

1

Experiments in Choice and Valuation

Non-Market Values for 

Alternative Operations of the 

Glen Canyon Dam: 

1

Hank Jenkins-Smith & Carol L Silva 

Overview

1. The Research Puzzle

2. Research Goals and Approach

3. Treatment #1: Replication

4. Treatment #2: Randomization of Information

5. Treatment #3: Structuring Alternative Options

6. Treatments #4-6: Tribes and Western Communities

7. Treatment #7: Hypothetical Air Emissions Effects

8. Summary and Implications

2
4/27/2015

2

The Puzzle

• Hydropower is broadly perceived to be a 

stable, domestic, and renewable form of energy

• Yet studies of social preferences indicate a 

willingness to pay to reduce reliance on 

hydropower

– Why is that?

– Do such studies miss important value dimensions 

when attempting to measure public preferences?

3

Preferred Future Energy Sources

• Asked in May, 2012: “Now think about the overall mix of all 

sources of energy for the U.S. We currently get about 83 

percent of our energy from fossil fuels, 9 percent from nuclear 

energy, 3 percent from hydroelectric dams on rivers and 

lakes, and 5 percent from other renewable sources (wood, 

biofuels, wind, waste products, geothermal, and solar).We 

want to know approximately what percentage of the total U.S. 

energy supply over the next 20 years you would like to see 

come from each of these four sources.”

– What percent of our energy should come from [fuel type], 

which currently provide about [XX]% of total U.S. energy? 

4
4/27/2015

3

Average Preferred 2032 Energy Mix

5

• What percent of our energy should come from hydroelectric dams on 

rivers and lakes, which currently provide about 3% of total U.S. energy?

• What percent of our energy should come from renewable sources (wood, 

biofuels, wind, waste products, geothermal, and solar), which currently 

provide about 5% of total U.S. energy? 

Perceived Energy Attributes by Source

Coal Oil Nuclear Natural 

Gas

Hydro Wind Solar

Clean 6.9% 7.5% 31.5% 42.1% 77.4% 85.7% 85.4%

Renewable 11.7 10.2 25.5 21.3 65.3 77.2 73.3

Safe 23.1 24.4 13.8 35.4 64.2 72.9 77.9

Plentiful 31.3 21.9 31.9 41.2 44.3 49.7 52.7

Preferred 9.6 8.8 16.7 31.3 56.9 60.2 64.8

“Now we want to know what kinds of qualities you associate with different 

ways of reliably generating large quantities of electricity for industries, 

businesses, and homes. Please check the descriptions you associate with 

each of the following sources of energy.”
4/27/2015

4

Overview

1. The Research Puzzle

2. Research Goals and Approach

3. Treatment #1: Replication

4. Treatment #2: Randomization of Information

5. Treatment #3: Structuring Alternative Options

6. Treatments #4-6: Tribes and Western Communities

7. Treatment #7: Hypothetical Air Emissions Effects

8. Summary and Implications

7

Research Goals

• Analyze issues related to valuation of 

nonmarket effects of changing Glen Canyon 

Dam operations

• Assess sensitivity of previous estimates of 

WTP for effects of changing operations to:

– Presentation of information

– Approach used to structure choices

– Inclusion of omitted dimensions of value
4/27/2015

5

Research Approach

• Attempt to replicate 1995 Welsh et al. study

• Then implement experiments that can be 

compared to replication baseline:

– Presentation of information

– Structuring of choices

• Allow for “negative valuation” of changing dam 

operations

– Can also be thought of as positive valuation for maintaining 

dam operations

– Introduction of omitted dimensions of value

9

Experimental Treatments

10

Treatment Concept Sample Size

1. Replication Used Welsh et al (1995) language and format, in 

Internet survey mode 341

2. Adds randomized 

presentation of info. to #1

Replication with the exception of randomizing 

the order of information 345

3. Adds referendum choice 

options to #2

Permits respondent to vote for one of two 

options, rather than one “proposal” 352

4. Adds Native American effects 

to #3

Adds information and potential effects of 

changed operations on Tribes 343

5. Adds rural Western 

community effects to #3

Adds information and potential effects of 

changed operations on rural communities 397

6. Combines #4 and #5 Combines information and potential effects on 

Tribes and rural communities 345

7. Adds hypothetical air 

emission effects to #3

Includes hypothetical effects of increased air 

emissions (not combined with 4-6) 342
4/27/2015

6

Research Approach

• National survey of 2,465 individuals

– Internet-based and weighted to Census

11

Collected: April 22-24, 2014

Average time: 26 minutes

Participant Source: SSI “Dynamix”

Panel

Respondent Mix: Close to Census

Weights: 2013 Census

Estimates

Overview

1. The Research Puzzle

2. Research Goals and Approach

3. Treatment #1: Replication

4. Treatment #2: Randomization of Information

5. Treatment #3: Structuring Alternative Options

6. Treatments #4-6: Tribes and Western Communities

7. Treatment #7: Hypothetical Air Emissions Effects

8. Summary and Implications

12
4/27/2015

7

Treatment #1 • Replication of Welsh et al (1995) approach

– Introduce purpose of survey

– Background information

• Provided a map of the study area

– Overview of natural resources in the study area

– Concerns about the resources

– Projected effects of changing operations

– Solicit views on changing operations

• Value preference more than $25?

• Online survey images (desktop, tablets, smart phones)

13

14
4/27/2015

8

15

16
4/27/2015

9

17

18
4/27/2015

10

19

20
4/27/2015

11

Replication Results

Welsh et al. (1995) Study Our Study

Vote for proposal to 

change operations 71% 70.6%

Not vote for proposal to 

change operations 17% 17.3%

Would not vote 12% 12.2%

21

• Nearly identical results

– Provides a baseline for exploring effects of:

• Presentation of information 

• Structuring of alternatives

• Additional value dimensions

Overview

1. The Research Puzzle

2. Research Goals and Approach

3. Treatment #1: Replication

4. Treatment #2: Randomization of Information 

5. Treatment #3: Structuring Alternative Options

6. Treatments #4-6: Tribes and Western Communities

7. Treatment #7: Hypothetical Air Emissions Effects

8. Summary and Implications

22
4/27/2015

12

Overview

1. The Research Puzzle

2. Research Goals and Approach

3. Treatment #1: Replication

4. Treatment #2: Randomization of Information

 No effect on WTP

5. Treatment #3: Structuring Alternative Options

6. Treatments #4-6: Tribes and Western Communities

7. Treatment #7: Hypothetical Air Emissions Effects

8. Summary and Implications

23

Overview

1. The Research Puzzle

2. Research Goals and Approach

3. Treatment #1: Replication

4. Treatment #2: Randomization of Information (no effect)

5. Treatment #3: Structuring Alternative Options

6. Treatments #4-6: Tribes and Western Communities

7. Treatment #7: Hypothetical Air Emissions Effects

8. Summary and Implications

24
4/27/2015

13

Treatment #3: Structuring 

Alternatives

• Original study structured choice as referendum 

on a proposal for changing dam operations

– Focuses valuation on the proposed change

• We structured choice as referendum allowing 

for selection between one of two options

– Conceptually more appropriate

– Necessary for estimation of positive values placed 

on maintaining dam operations

25

26
4/27/2015

14

27

Results

Welsh et al. 

(1995) Study

Our 

Replication

Two Options 

Treatment

Vote for to change 

operations 71% 70.6% 67.8%

Vote to maintain

operations 17% 17.3% 21.3%

Would not vote 12% 12.2% 11.0%

28

• Results indicate that one respondent in five will vote 

to maintain current dam operations

• This approach permits assessment of the value 

placed on not changing dam operations
4/27/2015

15

29

Results

Baseline-Replication 

of Welsh et al.

Two Options 

Treatment

Vote to change operations 70.6% 67.8%

Would you pay $25? NA 46.9

Vote against proposal (i.e. to 

maintain operations) 17.3% 21.3%

Would you pay $25? NA 6.9

Would not vote 12.2% 11.0%

30

• Nominally larger number of respondents chose to leave dam 

operations unchanged when offered two options

 6.9% would pay $25 to leave operations unchanged

• Appears to be a subtle but important effect:

 A single “proposal” appears to privilege the proposed change
4/27/2015

16

Overview

1. The Research Puzzle

2. Research Goals and Approach

3. Treatment #1: Replication

4. Treatment #2: Randomization of Information (no effect)

5. Treatment #3: Structuring Alternative Options

6. Treatments #4-6: Tribes and Western Communities

7. Treatment #7: Hypothetical Air Emissions Effects

8. Summary and Implications

9. Next Steps

31

Treatments #4-6: Omitted Value 

Dimensions

• Welsh et al. study valued only selected effects of 

changing dam operations

• Additional categories of nonmarket values may 

affect respondents’ choices (Loomis 2014): 

–Paternalistic altruism:

• Effects on Native American tribes

• Effects on rural Western communities

–Externalities:

• Effects on air emissions

– Relevant to general discussions surrounding 

hydropower

32
4/27/2015

17

Omitted Value Dimensions

• Experimental treatments (random assignment):

– #4: Native American tribes

– #5: Rural Western communities

– #6: Native American tribes + rural Western 

communities

• Information on these topics was provided in 

background material and presentation of 

alternatives for operating the dam

– Introduced in random order

– Example: Native American treatment

33

Background Information

34
4/27/2015

18

Concerns

35

Effects

36
4/27/2015

19

Alternatives

37

And for rural Western Communities...

For effects on Native American tribes...

Results

38

Replication 

With Two 

Options 

Native 

American 

Rural Western 

Communities

Native 

American + 

Rural Western 

Communities

Vote to change

operations 67.8% 49.5% 49.1% 32.4%

Would pay $25 46.9 33.0 33.9 21.8

Vote to maintain

operations 21.3% 39.2% 37.1% 57.1%

Would pay $25 6.9 16.5 16.4 30.3

Would not vote 11.0% 11.4% 13.8% 10.5%
4/27/2015

20

Results

• Paternalistic altruism can have a large effect 

on support for maintaining dam operations

• Effect is cumulative when Native American 

and Western community treatments combined 

into single treatment

– Nearly 60% of respondents support maintaining 

operations

– Over 30% willing to pay at least $25 to maintain 

operations

• Strong evidence for “Negative” WTP

• Would be treated as $0 valuation in most CV studies, 

including Welsh et al. study

39

Overview

1. The Research Puzzle

2. Research Goals and Approach

3. Treatment #1: Replication

4. Treatment #2: Randomization of Information (no effect)

5. Treatment #3: Structuring Alternative Options

6. Treatments #4-6: Tribes and Western Communities

7. Treatment #7: Hypothetical Air Emissions Effects

8. Summary and Implications

40
4/27/2015

21

Treatment #7: Air Emissions

• Added as pilot study with a hypothetical case

– GCD changes may or may not affect air emissions

• Relevant to debates about whether to build or 

remove dams that provide hydropower

– Net impact on distribution of sources of electricity

• Stand-alone treatment only

– In targeted application, would need to include 

other relevant non-market value dimensions

41

Background Information

42
4/27/2015

22

Concerns

43

Hypothetical Effects

44
4/27/2015

23

Alternatives

45

Results

46

Replication With Two 

Options 

Air Emissions

Vote to change operations 67.8% 36.7%

Would vote to pay $25 46.9 22.4

Vote to maintain operations 21.3% 51.0%

Would vote to pay $25 6.9 27.8

Would not vote 11.0% 12.3%
4/27/2015

24

Results

• Air quality dimension has large effect on 

support for maintaining dam operations

– Over half of respondents voted to maintain 

operations

– Over a quarter said they would pay at least $25 to 

do so

• Substantial implications for policy analysis

– Adding, upgrading generators and capacity

– Removing generators or dams

47

Overview

1. The Research Puzzle

2. Research Goals and Approach

3. Treatment #1: Replication

4. Treatment #2: Randomization of Information (no effect)

5. Treatment #3: Structuring Alternative Options

6. Treatments #4-6: Tribes and Western Communities

7. Treatment #7: Hypothetical Air Emissions Effects

8. Summary and Implications

48
4/27/2015

25

Summary of Findings

• Structuring alternatives as a referendum choice 

between two options allows respondents to 

consider bundles of distinct value attributes

– In contrast to Welsh et al (1995) on single proposal 

for changing operations

• A single proposal appears to privilege that proposal

– More appropriate conceptually in that options may 

require tradeoffs across different attributes

– Allows for expression of “negative” valuations

• Meaningful proportion of respondents place positive 

value on maintaining current dam operations

49

Summary of Findings

• Paternalistic altruism considerations appear to cause 

respondents to be substantially more likely to support 

maintaining current dam operations

– Effects on Native American tribes

– Effects on rural Western communities

– Combination of the two

• Respondents place significant value on low-emissions 

characteristics of hydropower

– Large implications for debates over adding/removing dams, 

or upgrading generator capacity

50
4/27/2015

26

Implications

• Suggests that respondents have positive WTP 

for maintaining operations when accounting 

for additional relevant nonmarket effects

– In stark contrast to Welsh et al. (1995) findings

– Phase II experiments do not permit estimation of 

population WTP 

• Will require full cv study 

• Appropriate treatment of “negative” values in 

calculation of WTP will require development of new 

estimation techniques

51

52

Hank Jenkins-Smith

hjsmith@ou.edu

Carol L Silva

clsilva@ou.edu

Center for Energy, Security, and Society

http://cess.ou.edu