Appendix 1: Review Comments and Federal Register Comments Received

June 10, 2013

To: Matt Weber

From: Kerry Smith

Subject: Arizona River Valuation Survey

As promised, I went through the survey. It is a very interesting project. I was really impressed by all 

of the documentation developed in planning the survey. There were a number of other aspects I liked in 

the research design. However, I will focus my comments on the issues where I felt there was some need 

for changes. I have listed them in order of importance: 

(1) The payment vehicle is not explained well or justified. Which taxes will go up and why? 

Why will the water be reduced in the “expected future”? You suggest on page 6 it is to 

reduce the amount of water used by the river ecosystem. What is the alternative use for 

the water that is not going to the ecosystem services? This is not explained. How will the 

proposed changes be maintained and why does it cost more to maintain a system that is 

the current baseline (without the water quality improvements)? These issues are not 

adequately explained. Until they are, as a reader I don’t know how or why this ends up 

increasing my taxes indefinitely. People know that taxes depend on income –how do you 

know what they will pay before you know their income? I realize in a mail question it 

cannot be dynamic in the way envisioned by this comment. Nonetheless it suggests the 

tax payment vehicle must be explained carefully.

(2) There are too many choice panels. I would explain the number respondents get and have 

no more than two or three. Respondents are not told each panel is independent of the one 

before. Since it is expected to be mailed, you need to allow in your experimental design 

that they will read ahead. It is essential that they cannot find the same option or a better 

option at a cheaper cost. With most conventional experimental design software programs, 

the blocking process for selecting choice sequences will not include this constraint. I have 
had to manually inspect all design panels in previous studies to be sure “better” options 

are not included as part of the randomization process.

I realize that this affects the assumption of randomization but respondents read ahead. It 

is the bundle of choice sets you give them that have to be considered as an integrated 

whole.

(3) I felt the choice set panels were too dense. I also felt “expected future” needs to be more 

carefully identified as what will happen if they do nothing. This is too implicit for me. 

Also, when is the expected future? What is the time horizon for this to happen? The 

timing and details on page 6 are too vague.

(4) A key element is using these conjoint or CV questions is that respondents believe their 

answers will be consequential. This follows from the Carson-Groves argument in ERE 

and Vossler and others recently in the AEJ supporting it. Your discussion on page 1 

gives the impression opinions will be used in selecting “best” options. I believe the 

statements need to be stronger – results based on respondents’ votes will be presented to 

policymakers and their decisions consider constituents desires. Explanation must be 

consistent with alternative uses for the water that are motivating the change from the

status quo.

(5) For me, the description of ecosystem services and situation with and without flow is 

inadequate. All one knows is the new miles of river and forest acres that is it. A little text 

that there are other “stuff” affected but little details.

(6) The term “Full body contact” in my opinion is terrible. Explain what you mean in plain 

English. Water will never be deep enough for swimming, but one could wade, raft and 

not be concerned if you felt in and went underwater. You also need to be specific about 

current water quality, fish live there, some other species are supported, “no full body

contact” is inadequate.

(7) As you note reminding respondents what to consider on page 12 – telling them the 

questions display “different tradeoffs” is inappropriate. Questions display alternative 
choices – don’t tell them to make tradeoffs. We assume they do but that is an assumption 

on our part.

You say there are three voting questions. I saw four plus an example. None of the 

explanation on this page was adequate. It really needs to be re-worked.

(8) You indicate in your discussion of recreation trips per year. You need to give the context 

in the text – trips in a typical year or trips last year; where do they go most often?

(9) Question 14 is too direct – you are using economic explanations rather than what people 

might use to explain their motivations.

(10) I would want varialbes to characterize respondents:

 education

 employment status

 own a home

 boat ownership

 more income categories (and leave out “estimated”)

 define what you mean by income and then ask them to give their best assessment

I realize most of my suggestions are written in fairly direct terms. I don’t mean them to be 

critical of you—I just wanted to emphasize their importance. I also wrote this quickly.

Finally, I did not get into issues with the application of that conjoint framework for valuation and 

the implicit assumptions about estimating WTP versus MWTP for changes in an attribute. You did not 

have much detail on this in what you sent. It might be worth sketching what you hope to derive from 

the model for policy analysis before finalizing the survey.
Arizona Stream Issue Questionnaire

Issues and Questions

John Hoehn

June 23, 2013

The questionnaire overall is very well put together. The text is readable and information. The sequencing

of sections and questions is logical. The diagrams and graphics are clear and helpful in understanding the

text and issues.

The key questionnaire is how your respondents react to and understand the issues in the questionnaire. I

assume that you’ve either tested or have plans to test the questionnaire with respondents like those in your

sample. With that assumption out of the way, I do have some comments on the questionnaire. None are

game changers but they are offered as issues to consider before testing and finalizing the questionnaire.

Here they are:

1. Page 3. What is treated wastewater? What’s meant by “this was just a method of disposing of”?

I think I understand, but I’m not sure. I suppose treated wastewater is sewage that’s been 

through some type of treatment facility? How much treatment? Is it discolored and 

smelly? Does it contain harmful bacteria, viruses or chemicals that may hurt plants, 

animals or me if I come into contact with it? Is there another term that could be used—

recycled water maybe?

2. Pages 3 to 12 are a little much in terms of explanation overload. The 10%/90% information

seems unnecessary to selection the options. Page 5 doesn’t seem relevant either. Page 6 

could be edited to reduce the amount of info and focus attention on options and issues in 

a choice table. The Expected Future lays out what you get for no additional cost. Options 

A and B are also possible if people are wtp. The Expected Future and the Options differ 

in the following:

a. Miles of streambed flooded

b. Acres of cottonwood/willow ecosystem

c. Whether the water is safe enough for full body contact

d. How the N and S are affected

e. Cost

The cost issue is somewhat confusing to me. How does one get more water by simply 

payingmore? Water has to come from someplace—will it raise water prices where I live? 

Can you offer a little more explanation?

3. Page 7. It seems like what’s represented is with and without water flow. Why not just label the

scenes accordingly—With and Without?

4. Page 9 Why is a sample table necessary? Respondents have probably already read ahead and

know what’s coming. Respondents typically skim through a questionnaire before 

deciding to take the time to answer it.

5. Page 13-15 Option B in page 15 seems like a real bargain compared to option A on page 16 

when I’m indifferent to north or south flows.

This raises a question—is the N/S question issue really important enough for the added

complexity. Water for the south seems cheap which seems a little surprising with Tucson in the

N. Why is water for S so much cheaper than water in N?

Would it be better to focus the cost/service tradeoffs on total flow miles, forest acres and body 

contact while leaving the N/S question as a separate allocation question? Are there returns to

scope or scale in provision that make the N/S question significant? If not, it might be better to go

with a simpler structure by eliminating separate rows for N and S flows, acres and body contact.
Hi Matt,

As we discussed on the phone just now, Emily and I are very supportive of your work on the “Willingness 

to Pay” survey focused on the Santa Cruz River. The results of this work will be very helpful for Sonoran 

Institute’s Santa Cruz River Initiative. Understanding community values regarding natural resource 

management is critical for prioritizing on the ground actions, crafting policy, leveraging additional funds.

We found this version of the survey to be much improved from the earlier draft you shared. We have 

some specific comments/edits regarding the language in the survey that I list here in order by page:

 Page 2 – the italicized intro language makes it seem like these are real options that authorities 

are considering. We suggest a rewording of the third sentence. Instead of “Your responses 

will help authorities to select the best option.”...we suggest something more like: Your 

responses will help authorities to better understand public opinion regarding management of 

the river.

 Page 6 – the language is misleading regarding the cut back to river flows in the “expected 

future”. Seems river flows would be reduced because managers think the river uses too 

much. Rather flows will be reduced because effluent may be used for other purposes. Instead 

of “In the future, river flows are expected to be cut back to reduce the amount of water used 

by the river ecosystem.” ....perhaps something more like this: In the future, river flows are 

expected to be cut back as effluent is used for other purposes rather than released into the 

riverbed. To me this is more accurate.

 Page 6 – reference to voting in the Flow & Forest description implies that the outcome is 

really up to the survey respondents.

 Page 9 – provide an explanation as to why certain boxes within the questions are shaded. We 

assumed these were items that did not change from the expected future. This should be more 

explicitly stated.

 Page 12 – there is a typo in the first sentence. There are 5 voting questions, not 3. 

 Page 12 – the language in the third bullet referencing voting again implies that these are real 

options that people are voting for. Perhaps reword to say something like: Review and answer 

each question carefully. Understanding your opinion will help authorities to best manage the 

river.

 Page 18 – minor formatting typo under question 9, first bullet. The text “Never chose” is not 

the same font as the subsequent questions.

Thank you for the opportunity to review and comment on this work. We would be happy to discuss this 

work further with you.

Claire and Emily

Claire Zugmeyer

Ecologist
The Pima County Regional Wastewater Reclamation Department appreciates the opportunity to 

comment on the proposed EPA public opinion survey (EPA-HQ-ORD-2013-0282-0001) regarding future 

options for the water quality, water quantity, and other environmental benefits of the effluentdependent reaches of the Upper and Lower Santa Cruz River. Our comments will focus on the Lower 

Santa Cruz River as that is the reach managed by various Pima County Government entities and utilized 

for effluent discharge and aquifer recharge by this Department. 

The survey approach is to present the public with several potential future outcomes for the water 

quality, water quantity and habitat (“forest cover”) of a reach of the River (with photos and graphic 

illustrations) and ask that survey recipients choose from several incremental, per-household tax 

amounts that participants would be willing to pay for these outcomes. Although we are very interested 

in public perception and opinion related to water quality of this effluent-dependent portion of the 

Lower Santa Cruz (LSC) River, we nevertheless have significant concerns about the survey methodology 

and approach, background information provided, and the alternatives presented to the public in this 

survey. Highlights of our concerns are as follows:

 The survey approach assumes that the outcomes listed are feasible given the amount of 

funding stated. In fact, the legal, scientific and physical circumstances of these effluentdependent flows combine to make many of the outcomes highly unlikely (if not nearly 

physically impossible), such as the river to meet “full-body contact” water-quality

standards. For more information about the complex and unique issues of water quality 

and water quantity for effluent-dependent waters in the arid west, we recommend that the 

survey personnel review the Arid West Water Quality Research Project, which took place 

from 1995 to 2007 and which was funded and supervised by EPA Region IX. If 

necessary, we can provide copies of reports from this effort. 

 The survey does not provide adequate background to the public as to the nature and 

impact of the over $600 million in water-quality improvements to its two major 

metropolitan Water Reclamation Facilities (WRFs) that Pima County is currently 

constructing in coordination with the Arizona Department of Environmental Quality. 

These facilities provide the effluent flow in this reach of the Lower Santa Cruz River and 

will have state-of-the-art water-quality treatment processes and advanced odor control for 

which Pima County ratepayers are totally responsible for paying through increased sewer 

user charges. We can provide more details about this effort.

 The future alternatives presented in the survey neglect the economic and environmental 

benefits of using the water beneficially off-channel as reclaimed water or in-channel for 

aquifer augmentation.

 The survey approach includes sending the survey to selected residents in the Phoenix and 

Tucson Metropolitan areas, but the effluent-dependent flows of the Santa Cruz do not 

extend to the Phoenix areas and those residents are generally unaware of the River at all. 

Furthermore, people living along the Upper Santa Cruz River in Santa Cruz County are 

not surveyed. Therefore, half of the targeted survey recipients who have no connection to 

the Santa Cruz will be surveyed, while others with a direct connection will not.
In closing, the Department would recommend that, prior to proceeding further with this survey, EPA 

staff discuss the survey formally and fully with the Pima County Government management and staff to 

reconcile these and other discrepancies. We feel that addressing these concerns will lead to an 

improved survey and research study.