Document ID: EPA-HQ-ORD-2013-0282-0013
Agency: epa
Document Type: Supporting & Related Material
Title: 
Posted Date: 2013-11-27T05:00Z

Response to Comments Received After Second Federal Register Notice
1) The first commenter describes in detail the policy background for the South reach of the Santa Cruz River, stressing lack of control that Arizona has over most of that effluent, which is actually owned by Sonora, Mexico. The commenter expresses support for the survey research as an opportunity to proactively evaluate options. There were no statements requesting a response.
2) The second commenter also notes the potential for changes to the South reach of the Santa Cruz River, and the importance of gaining an accurate sense of how residents and tourists value the natural resources. The commenter states that a monetized value elicited by the survey should not be the only basis for assessing value. The principal investigator agrees that the survey should be only one of numerous considerations and sources of information to support local decision-making. 
3) The third commenter submitted a comment after the comment period closed. The commenter references in some places a conversation held with the principal investigator (PI). As further background, the principal investigator (PI) became aware of the concerns of this commenter late in the comment period and phoned a representative immediately to discuss the research project since the public review period was close to ending. The PI furthermore offered to be available for follow-up conversations while the commenter reviewed the project more closely, but the PI did not hear from the commenter again. Unfortunately the single conversation was incomplete and had led to some misunderstandings, evident in mischaracterization of the research and mischaracterization of statements the PI made. 
While below there are detailed responses to the specific comments, an overall response may also be useful to describe the project and its purpose. The objective of the survey is to contribute to research on how people value river ecosystems in the arid southwest, with the Santa Cruz River as a case study in this regard. The survey and survey design reflect an extended period of careful study of Santa Cruz River management options and pretesting of the survey instrument, as described at length in the supporting statement (see especially Part B, section 3). The survey is an example of best practice methods in willingness to pay survey research to estimate value for the scenarios posed. The survey strategically considers a range of effluent management possibilities for the Santa Cruz River and is not intended to provide a complete cost-benefit analysis of any particular Santa Cruz River effluent management option. The economic information the survey would provide is designed to be useful for management insight, but would be incomplete for any given effluent management option for several reasons: the willingness to pay survey design considers a discrete set of marginal changes for a discrete set of variables; the survey focuses on urban populations; and there are several additional types of economic analyses possible, such as estimating the cost of increasing the level of treatment to actually allow safe full body contact.
The commenter raises several good questions, some of which are commonly asked of willingness to pay survey research methods. Some of the responses will duplicate responses made to comments received in the first round of comments found in the supporting statement part 3(c) or other material provided in the supporting statement. The full text of the comments is responded to below, organized by four points in the federal register that EPA requested comments on. EPA solicited comments and information to enable it to: (1) Evaluate whether the proposed collection of information is necessary for the proper performance of the functions of the Agency, including whether the information will have practical utility; (2) evaluate the accuracy of the Agency's estimate of the burden of the proposed collection of information, including the validity of the methodology and assumptions used; (3) enhance the quality, utility, and clarity of the information to be collected; and (4) minimize the burden of the collection of information on those who are to respond, including through the use of appropriate automated electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses.
EPA Question 1 for commenters: 
Evaluate whether the proposed collection of information is necessary for the proper performance of the functions of the Agency, including whether the information will have practical utility.
Comment 1.1 from Tumacácori National Historical Park: 
We strongly urge the author to consider what we feel is a flawed emphasis on increasing survey response at the expense of facts; and also consider ways that this study may be used in the future as a tool in directing management action or in influencing public opinion in ways for which it is not designed or suitable. 
We understand the current fiscal restraints placed on federal agencies that challenge our capacity to achieve agency and operational objectives. As a fellow federal agency, we understand the conflict between these challenges and desire to get the job done. However, we strongly object to including statements in the survey that the author knows are not factual to increase the response rate, and to limiting the distribution of the survey to the Tucson and Phoenix metropolitan areas strictly to assure an adequate sample size. 
Response to Comment 1.1 from Tumacácori National Historical Park:
The comment mischaracterizes the survey research as an attempt to mislead survey respondents. The PI believes the commenter is referring to the consequentialist language in the survey, a best practices technique in willingness to pay survey design. Consequentialist language emphasizes that respondents' survey responses matter. This is not a response rate issue but an issue of limiting potential bias in responses due to strategic behavior. This issue is treated in more depth below in the more specific comment made later, comment 1.5. 
One of the motivations of the survey is in fact that it may provide management insight. The survey is not a tool to influence public opinion, but rather a tool to proactively collect public opinion. 
The decision to sample Tucson and Phoenix are driven by the urban emphasis of the research project, not in order to assure an adequate sample size.  
Comment 1.2 from Tumacácori National Historical Park: 
Pursuant to this we offer the following points: 
   * The survey vastly oversimplifies the scenarios to the point that the resulting data will be of minimal value. For example, the survey does not adequately describe impacts of the various proposed scenarios, such as that the diversion of water away from the river will result in sharply declining groundwater levels throughout the upper Santa Cruz river basin. 
Response to Comment 1.2 from Tumacácori National Historical Park:
The scenarios in the survey were carefully refined and pretested over a lengthy sequence of meetings with urban residents in southern Arizona, in two phases. Preceding these pretests of the survey instrument were several focus groups collecting public perspectives on rivers and streams in southern Arizona, including the Santa Cruz River. 
The survey is not intended as a tool to estimate the value of changes in groundwater levels. For the flow & forest variable, the survey is designed to value changes in length of surface flow and acres of forest. 
Comment 1.3 from Tumacácori National Historical Park:
   * There is no indication that the scenarios included in the survey are economically feasible or even physically possible, but the scenarios are posed without discussion of these uncertainties. For example, recent upgrades to the Nogales International Wastewater Treatment Plant (NIWTP) have greatly improved the water quality of the effluent, especially the nutrient levels. The NPS supports this and any other initiatives to improve the water quality. There is currently, and likely for the near future, unique features of the waste stream entering the NIWTP that make assuring that the water is safe to swim, or even wade in, nearly impossible. Much of the source of the effluent at the NIWTP if from Nogales Mexico, and is thus outside the jurisdiction of American environmental regulation. Water in the waste stream from Nogales Mexico and effluent from the treatment plant has been detected with concentrations of metals such as chromium, cadmium and lead that exceed Arizona and federal standards. Mitigation of theses metals and other compounds, like emerging contaminants (e.g., pharmaceuticals) detected in the effluent is currently not possible -- treatment is cost prohibitive or not currently feasible.  
Response to Comment 1.3 from Tumacácori National Historical Park:
That source waters of the Santa Cruz River cross jurisdictions does not mean that changing the water quality of the Santa Cruz River is not a management possibility. The Arizona Department of Environmental Quality, Office of Border Environmental Protection (OBEP) website states (http://www.azdeq.gov/obep/, accessed 11/25/2013): "OBEP's emphasis is on cross-border or transboundary issues that impact Arizona's environment and its citizens. This entails working in a binational and bicultural setting to facilitate efforts aimed at improving air quality, waste management and water quality conditions in Arizona border communities." The scenarios were built to reflect changes in ecological attributes of high interest to the public (as collected through the focus groups and pretesting) and were also constructed to be plausible. Regarding the safe full body contact option described in the survey, it would not be possible to know if further effluent treatment or pollutant prevention measures are cost-effective without having a sense of what public values for increases in water quality are.  
The survey does not attempt to include actual costs for the scenarios posed. Instead the payment levels in the choice experiment design are derived from survey pretesting, and are designed to bracket willingness to pay for the different scenarios. These payment levels would be updated based on the pilot survey (as described in the survey methodology supporting statement Part B, section 3) to refine their ability to bracket willingness to pay. Estimating costs of the scenarios is a different research question that would require different techniques.
Comment 1.4 from Tumacácori National Historical Park:
   * The principal investigator has verbally indicated to my staff that the purpose of the study is to "understand how people value rivers in the southwest" (in general). However, this study specifically addresses two effluent-dominated reaches which are in no conceivable way representative of the majority of rivers in the southwest, and the responses to the survey will not be relevant to the wider question of southwest rivers.
Response to Comment 1.4 from Tumacácori National Historical Park:
The objective of the survey is to contribute to research on how people value river ecosystems in the arid southwest, and the Santa Cruz River is a case study in this regard. Indeed there are a number of important differences between the effluent-dominated reaches of the Santa Cruz River and other rivers and streams in the southwest. For a variety of reasons it would not be possible to take the results from this study and apply them directly to management scenarios for any river in the southwest. However there may be insights for similar management contexts in the arid southwest beyond the case study. The techniques for utilizing valuation survey results for new contexts are a topic in the "benefit transfer" literature. A full discussion of that literature and those techniques is best left to other sources. The research report would clearly describe the management context for the case study: this information would be important both for interpretation of the case study results, as well as considering how the results may or may not provide insight into other management contexts.  
Comment 1.5 from Tumacácori National Historical Park:
   * The survey indicates that a vote in this survey will lead to subsequent relevant management action. According to the principal investigator, this is not the case. The non-binding and pure research nature of this survey should be clearly indicated and any text/graphics with this inference (e.g., pg. 6) should be removed or altered. It is stated that on page 11 of the survey instrument that, "This survey is similar to a public vote". This statement leads the survey participant to believe that the results of this survey are somehow formal in nature and will lead to a management action when in fact there is no "vote" being taken nor are any results being immediately considered by any management agency as a result of this survey, as per conversations with the author. According to the principal investigator, the survey is for academic research purposes with a goal of publication in a scholarly journal and was not conceived to specifically or primarily affect management decisions. This fact should be clearly and explicitly stated. 
Response to Comment 1.5 from Tumacácori National Historical Park:
While there is no specific management action this survey is designed to address, the text in the survey is designed to be consequential for respondents. This follows best practices of willingness to pay techniques (Carson and Groves, 2007, Environ Resource Econ, 37:181-210; Vossler et al., 2011, American Economic Journal, 4(4): 145-171). This is not an issue of response rate (although low response rates increase the danger of non-response bias) but in order to limit biased responses due to strategic behavior. Unrealistically high willingness to pay choices could be made if respondents don't believe they will actually have to bear the costs they are indicating a willingness to pay for. Consequentiality cannot be sacrificed without losing the goal of the survey to estimate values for the scenarios posed. The study is being designed to the standards of peer-reviewed literature, not to distance its usefulness for policy applications, but to ensure its rigor. It is a mischaracterization of the research to say that it is not designed to be informative for management decisions; however it is accurate to say that there is no single and specific management change the survey is designed to address. The following changes have been made to ensure that the survey is not viewed as an immediate actual vote, but also maintain consequentiality: Page 2, 1[st] paragraph: "Your responses will help authorities select the best option." to "This survey is designed to help authorities select the best option."; Page 6, 1[st] paragraph (two instances): "This survey will help managers know..." to "This survey is designed to help managers know..."; Page 13, 3[rd] bullet: "Your votes are important and will help determine the best option for the Santa Cruz River." to "Your votes are important and voting questions are designed to help determine the best option for the Santa Cruz River."; new question after Q10, "Thinking about how you responded in Q2 through Q5, would you choose the same options in an actual public vote?", with a five point scale having options of strongly disagree, disagree, neither agree nor disagree, agree, and strongly agree.
Comment 1.6 from Tumacácori National Historical Park:
   * The survey indicates that results will be used by "authorities" to "select the best option". It states that it is important to "hear from all perspectives", but the sample base constitutes major urban centers, only one of which is truly in southern Arizona. Rural and Santa Cruz County populations are not sampled. The principal investigator has stated that he assumes that residents of Santa Cruz County will have the same opinions as those of Tucson, but other comments he made indicate that this was done for ease of implementation alone. 
Response to Comment 1.6 from Tumacácori National Historical Park:
The consequentialist text on page 2, as described in the previous response, would read: "This survey is designed to help authorities select the best option." The importance of hearing from all perspectives is stressed early in the survey to ensure it is clear that no particular type of response (i.e. environmentalist or non-environmentalist) is sought. The survey research design in the supporting statement clearly states that the targeted samples are urban populations of the Tucson and Phoenix areas. The project design investigates urban values for the resources, and furthermore compares value estimates of an urban population close to the resources with an urban population distant from the resources. The research design has an urban emphasis, which is in influenced by the high percentage of persons in southern Arizona living in urban areas. If one considers all of the counties of southern Arizona (Cochise, Gila, Graham, Greenlee, La Paz, Maricopa, Pima, Pinal, and Yuma), the census-designated urbanized areas of just Tucson and Phoenix/Mesa account for 79% of the population (Census 2010, and http://www2.census.gov/geo/ua/ua_st_list_all.txt, accessed 11.25.2013). The urban emphasis of this project is not meant to indicate rural populations should not be studied in other research designs or are unimportant, but rather just denotes the boundaries of the study. The principal investigator has not stated that residents of Santa Cruz County will have opinions "the same" as those in Tucson. This false impression may have occurred in the course of describing that from an aggregate valuation standpoint, it is likely more informative to sample urban populations such as Tucson and Phoenix. The project has had an emphasis on investigating urban values from the beginning; the design boundaries of this study will be included in project reporting.
Comment 1.7 from Tumacácori National Historical Park: 
We understand that some current methods for conducting surveys suggest that response rates can be increased by making the respondent feel that by responding they can affect immediate change. We also understand the importance of a valid sample size to scientific studies. 
Tumacácori NHP continues to be a vital and vibrant focal point for celebrating the cultures and communities, past and present, associated with the Santa Cruz River valley. As an agency we take seriously our core mission of conserving our natural and cultural resources. We strongly feel that if the survey cannot be conducted using facts or even give a voice to the people that live along the resource, due to budget, manpower or time limitations, then it should not be conducted until the needed resources are available. 
We feel that although the survey would be useful, the attempts at sampling efficacy of the current survey have created a deeply flawed instrument, and in current form will not produce results that will be of real "practical utility". 
We also urge the author to consider that in the future, this study and its conclusions will be viewed by the public as scientifically rigorous and sanctioned by the EPA, a prominent federal agency tasked with protecting the environment. Whatever the outcome of the study, this association with the EPA will add a gravitas to the study that would not exist if it was conducted by another entity such as a university or an NGO with no government affiliation. The results of this survey may be used by any number of groups for scientific, community or political purposes that could have an impact on Tumacácori NHP and other stakeholders of the Santa Cruz River corridor. We urge extreme care in the design of this survey.
Response to Comment 1.7 from Tumacácori National Historical Park: 
The PI agrees that the survey expresses to respondents that their input matters. This is a common feature of survey research and was intentional.
The PI believes there is a role for collecting public input regarding management of public resources. To be proactive in collecting public input on a variety of potential effluent management actions the PI has constructed potential management scenarios and emphasized in the survey that public input is important in deciding among them. The PI is not in need of additional resources to conduct the research study as proposed. 
The PI has followed best practices for willingness to pay survey design and has extensively pretested the instrument to ensure its usefulness, including attention to comprehension of the survey text. Thus the PI believes the survey has high practical utility for estimating urban values for the scenarios posed.
The PI believes it is preferable to have public input on the effluent management issues considered in the survey, by use of a carefully designed and pretested instrument, than for public input to not be available. The PI agrees that its intent is a scientifically rigorous study. The PI views the potential for management insight as a motivation for, rather than a detriment of, the study. 
EPA Question 2 for commenters: 
Evaluate the accuracy of the Agency's estimate of the burden of the proposed collection of information, including the validity of the methodology and assumptions used.
Comment 2.1 from Tumacácori National Historical Park: 
We have noted several inconstancies as described in the survey and the supporting statement (EPA-HQ-ORD-2013-0282-0002): 
   * The estimate of the burden of the proposed collection does not include, or at least clearly indicate the development of the survey or the supporting documentation, the development of the notice to the Federal Register, or the time required by commenters to review these documents and develop comments. 
   * The estimate of burden to the agency covers a period of 20 weeks from the start of mailing the survey, and does not include development of the survey, the analysis or writing the report, but the estimate includes only 8 weeks of labor by the principal investigator. 
   * The methodology and the assumptions used to estimate the burden appear to misleadingly underestimate the actual burden. 
Response to Comment 2.1 from Tumacácori National Historical Park:
The estimates are correct as listed. The burden calculation in the supporting statement (part 6) provides information on the cost and burden associated with the information collection for both the respondents and the agency. The survey timeline extends over a period of 20 weeks since there are time gaps while people receive and reply to the survey, not all of this time would actually be dedicated exclusively to the collection of the data.
The survey was developed in association with research outcomes from an initial round of focus groups, and then pretesting of the survey instrument using focus group and interview techniques. The public burden of these focus groups and interviews is not included in the supporting statement for the survey but was applied for and approved, under the separate ICR# 2090-0028. This is stated in the supporting statement section 5(a). 
Comment 2.2 from Tumacácori National Historical Park:
   * The survey and comments do not sample representative populations that would be most affected by the proposed scenarios. Aside from the public register notice, no formal notice was provided to residents of Santa Cruz County, the location of the effluent-dominated upper Santa Cruz reach referenced in the survey, and none of the residents of this area are included in the survey. Very few comments have been received in response to the Federal Register notice, but a wider public awareness of this survey and the potential implications would without a doubt elicit many more comments on this controversial issue. 
Response to Comment 2.2 from Tumacácori National Historical Park:
This point does not address an alternate methodology for calculating burden. The intention to sample urban populations is addressed in response to comment 1.6 above. The intention of the survey is precisely to broaden the knowledge of public opinion on the issues the survey considers.
Comment 2.3 from Tumacácori National Historical Park:
   * If one purpose of the study is indeed to assess the value that people place on rivers in the southwest (as communicated by the author), non-effluent dominated rivers should be included in the scenarios, such as the Verde, the Gila, and the Colorado, to name a few. 
Response to Comment 2.3 from Tumacácori National Historical Park:
This point does not address an alternate methodology for calculating burden. As described in response to comment 1.4, the study is designed to be relevant to Santa Cruz River management, but there may be management insights on other similar contexts.  
EPA Question 3 for commenters: 
Enhance the quality, utility, and clarity of the information to be collected.
Comment 3.1 from Tumacácori National Historical Park: 
The quality, utility and clarity could be enhanced by consideration of the following points: 
   * Revise the survey significantly to include a balanced presentation of the outcomes of feasible alternatives. 
Response to Comment 3.1 from Tumacácori National Historical Park: 
This study, like any study, could have been designed differently and could have addressed additional questions. As designed it follows best scientific practice to address a limited set of meaningful questions and provide useful information on them. 
Comment 3.2 from Tumacácori National Historical Park:
   * The quality of information would be enhanced by sampling rural as well as urban populations and by increasing the number of respondents from southern Arizona as contrasted with respondents from central Arizona. 
Response to Comment 3.2 from Tumacácori National Historical Park:
EPA ORD has chosen a research design that emphasizes urban populations, a decision that was influenced by the disproportionate percentage (79% as described in response to comment 1.6 above) of the population of southern Arizona living in the urban areas to be sampled. The scope of the research is not meant to provide a complete accounting of effluent management options for all populations. Research in rural areas is not proposed by this research project. This is an important limitation to the study which will be noted in study reporting.  
Comment 3.3 from Tumacácori National Historical Park:
   * The flow scenarios should be more explicit in regards to where and how the various reaches of each of the wetted and non-wetted river reaches would be impacted and should provide clarity on the complexity of the issues related to the questions being asked. The survey questions and project background do not sufficiently describe the impacts that would occur as a result of the changes in management of the Santa Cruz River proposed in the scenarios. If the public were informed of these potential impacts, such as loss of biodiversity, loss of riparian areas, and stress to wildlife species at risk, the response would be different than it would otherwise be without that background. 
Response to Comment 3.3 from Tumacácori National Historical Park: 
A key element of the survey is partnering research allowing for the characterization of current condition flow and forest conditions, and overall predictions of how forest extent (for both the North and South Santa Cruz River effluent-dominated reaches) would change with changes in extent of surface flow. These natural science predictions were made available by researchers at the University of Arizona and Arizona State University, as described in the consultations section of the supporting statement part 3(c). However these models are considered preliminary and are not highly spatially explicit. In terms of information provision, the survey is intentionally conservative in describing the management scenarios. The survey language is specific about the changes posed and only includes information to the extent specific changes can be modeled. The information included was extensively pretested and designed to focus on the key variables. Explanations of increased extinction risk or actual extinctions for certain species were not included. The PI agrees that this kind of information, if it was available and if it was included, would likely influence survey responses. It is well-known that information provided in the background of a willingness to pay survey influences results; the results of this survey would need to be interpreted and represented in the light of the information included. 
Comment 3.4 from Tumacácori National Historical Park: 
   * There should be citations listed on the survey for the natural system models referred to in the survey, and to other relevant documentation (e.g., groundwater-riparian vegetation interactions). 
Response to Comment 3.4 from Tumacácori National Historical Park:
There were no requests from the pretest participants, drawn from diverse urban sociodemographics, to cite the basis for the natural science modeling in the survey background, although it would be appropriate to describe the methods and basis for other types of audiences. For example, the natural science models would be cited in the research report.
EPA Question 4 for commenters: 
Minimize the burden of the collection of information on those who are to respond, including through the use of appropriate automated electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses.
Comment 4.1 from Tumacácori National Historical Park:
We do not feel that this survey would be an undue burden to respondents, and the estimates by the author in the supporting statement in regards to respondent burden are realistic. 
Response to Comment 4.1 from Tumacácori National Historical Park:
The PI concurs.
Closing Comments from Tumacácori National Historical Park:
In conclusion, we feel that a survey to understand the value that people place in the Santa Cruz River would be a very useful tool. We also understand and laud the author's interest in conducting innovative and interesting research in social science within the region, and the likely usefulness of the results of this study to Tumacácori NHP. However, we feel that in the current form, the survey is flawed and will not yield useful results. 
We strongly recommend that the authors consider a survey that does not obfuscate the intent or the issues for the sake of sample design or efficacy, and consider the population with the greatest interest in the resource. Thank you for providing this opportunity to comment.
Response to Closing Comments from Tumacácori National Historical Park:
The PI appreciates the chance to hear these comments from Tumacácori National Historical and greatly respects the work of the Park. The PI has responded to the detailed comments in the appropriate sections above. The careful sequence of pretesting and guidance from the best practices literature on willingness to pay survey instruments have been chronicled or reiterated from the supporting statement. The PI believes the survey rests on an extremely strong foundation. The project has an urban emphasis and this emphasis would be clearly stated in the research report. This is not meant to indicate that complementary research should not be pursued on rural values, nor meant to indicate that the variety of related potential research questions on effluent management are unimportant. The PI has not obfuscated intents or issues but has instead transparently proposed a survey to collect public opinion on public resource management questions.