Air Pollution: Estimated Benefits and Costs of the Navajo Generating Station's Emissions Limit
Published: 1998-01-27T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a congressional request, GAO reviewed the Environmental Protection Agency's (EPA) decision to limit sulfur dioxide emissions from the Navajo Generating Station, focusing on: (1) the effect on emissions reductions and the associated costs that resulted from the negotiated agreement used by EPA in making its decision compared to its initial proposal; (2) the visibility improvements the agency estimated would result from the emissions controls and the means by which these improvements were determined; and (3) how contingent valuation was used to estimate the monetary value of visibility improvements.
section_title: What GAO Found
paragraphs: GAO noted that: (1) the negotiated agreement is expected to result in greater emissions reductions at less cost than EPA had initially proposed; (2) the agency initially proposed limiting sulfur dioxide emissions at the Navajo Generating Station by approximately 70 percent at an annual cost estimated between $91.9 million and $128.3 million; (3) the negotiated agreement is expected to increase emissions reductions to approximately 90 percent at an estimated annual cost of approximately $89.6 million; (4) the lower costs resulted from several factors, according to the plant operators; (5) according to a project engineer for the Salt River Project, with its compliance determined on an annual basis, the plant can operate its emission control equipment most days at a rate greater than that needed to cut emissions by approximately 90 percent to make up for those days on which emissions are not controlled because the equipment is not operating; (6) also, delaying the initial installation of the emission control equipment by almost 3 years, from January 1995 to November 1997, allows the project to be completed in a more cost-effective manner; (7) EPA estimated that reducing the sulfur dioxide emissions at the Navajo Generating Station by approximately 90 percent would improve winter seasonal average visibility at the Grand Canyon approximately 7 percent--from about 124 miles to about 133 miles; (8) most of this improvement was estimated to result from improvements during certain winter weather conditions; (9) EPA initially estimated an approximately 14 percent improvement in the winter seasonal average visibility primarily on the basis of a National Park Service study of visibility in the vicinity of the Grand Canyon; (10) EPA revised this estimate to approximately 7 percent after considering the results of other analyses; (11) however, EPA noted that its revised estimated may be understated because it did not include visibility improvements: (a) below the rim of the Grand Canyon; (b) in seasons other than winter at the Grand Canyon; and (c) year round at other nearby national parks; (12) both EPA and the Navajo Generating Station's owners used contingent valuation to estimate the monetary value of visibility improvements; and (13) although relying on the same methodology, the studies were different and yielded widely different results.

section_title: Letter
paragraphs: In 1989, the Environmental Protection Agency (EPA) made an initial determination that it was reasonable to expect that sulfur dioxide emissions from the Navajo Generating Station were contributing to impaired visibility in the Grand Canyon National Park, most notably during certain winter weather conditions. Although emissions controls for visibility impairment were not required when the coal-fired power plant became fully operational in 1976, subsequent amendments to the Clean Air Act require such controls if a source is found to be causing or contributing to visibility impairment in certain national parks and wilderness areas, including the Grand Canyon National Park. Accordingly, in February 1991, EPA initially proposed a rule requiring the Navajo Generating Station to reduce its sulfur dioxide emissions by approximately 70 percent. Subsequently, in October 1991, partially on the basis of a negotiated agreement between the plant’s owners and environmental groups, EPA issued a final rule that required an emissions reduction of approximately 90 percent. A project to install emission control equipment is under way, and, according to the plant operators, the project is on schedule. The first of three emission control units has been placed in service. On October 10, 1997, the plant operators notified EPA that the first emission control unit was operational and would comply with the approximately 90 percent emissions reduction beginning November 19, 1997, as required by the final rule. The overall project is scheduled to be completed by 1999.
Concerned about the benefits and costs of installing sulfur dioxide controls at the power plant, you asked us to review EPA’s decision to limit emissions. Following discussions with your office, we agreed to (1) determine the effect on emissions reductions and the associated costs that resulted from the negotiated agreement used by EPA in making its decision compared to its initial proposal, (2) identify the visibility improvements the agency estimated would result from the emissions controls and the means by which these improvements were determined, and (3) determine how contingent valuation was used to estimate the monetary value of visibility improvements. Contingent valuation is a methodology that relies on surveys to elicit information from consumers to estimate how much they would be willing to pay for something, including non-use values. While the contingent valuation methodology is controversial, it is currently the only known approach for estimating non-use values. We are not taking a position on the appropriateness of contingent valuation. Appendix I is an overview of the contingent valuation methodology.
subsections: section_title: Background
paragraphs: The Navajo Generating Station (NGS) is a 2,250-megawatt coal-fired power plant located near Page, Arizona. The plant, which became fully operational in 1976, is located approximately 12 miles from the northern boundary of the Grand Canyon National Park. The Salt River Project Agricultural Improvement and Power District (Salt River Project) operates the plant and owns 21.7 percent. Other owners and their shares are the Department of the Interior’s Bureau of Reclamation, 24.3 percent; Los Angeles Department of Water and Power, 21.2 percent; Arizona Public Service Company, 14 percent; Nevada Power Company, 11.3 percent; and Tucson Electric Power Company, 7.5 percent.
The 1977 amendments to the Clean Air Act set as a national goal “the prevention of any future, and the remedying of any existing, impairment of visibility” in certain parks and wilderness areas where such impairment results from man-made air pollution. The amendments include a requirement that sources with emissions “which may reasonably be anticipated to cause or contribute to any impairment of visibility in any such area, shall procure, install, and operate” the best available retrofit technology. In determining the emissions limit that reflects the best available technology, several factors are to be taken into account, including the costs of compliance, the energy impacts and impacts besides those on air quality, the remaining life of the power plant, and the degree of improvement in visibility that may reasonably be anticipated to result from the use of the technology.
section_title: Negotiated Agreement Provides for Greater Emissions Reductions at Lower Cost Than Initially Proposed
paragraphs: EPA’s final rule to limit emissions from NGS relied on the details of a negotiated agreement, between the power plant owners and environmental groups, which EPA expects to result in greater emissions reductions at a lower cost than EPA’s initial proposal. The agreement increased the level of emissions reductions from EPA’s proposed 70 percent to 90 percent, with estimated annual costs dropping from between $91.9 million and $128.3 million to $89.6 million. The amount of emissions that would be removed annually is expected to increase from about 50,000 tons of sulfur to about 64,000 tons. The negotiations included officials representing the owners of the plant, environmental groups, the state of Arizona, and EPA. These officials recommended the negotiated agreement to EPA as an alternative to the agency’s initial proposal to reduce the emissions from the power plant.
subsections: section_title: EPA Requested Comments on Alternatives to Reduce Emissions
paragraphs: In February 1991, EPA solicited comments on a proposed rule laying out a variety of strategies to reduce emissions from the power plant. EPA explained that, because of the uncertainty in determining the improvement in visibility expected as a result of limiting the emissions, it was considering and sought comments on four options to limit these emissions—a 50-percent reduction, a 70-percent reduction, a 90-percent reduction, and allowing the plant owners to test alternative technologies and select one if it met minimum emissions reductions at a set cost. In addition to the four options, EPA also solicited comments on any other appropriate alternative to limit sulfur dioxide emissions, such as controls used only on a seasonal basis. EPA’s proposed 70-percent emissions limit was the same as the standard the agency used at the time for new facilities. EPA estimated that a 70-percent emissions reduction would eliminate about 50,000 tons of sulfur from the power plant’s emissions annually and that the cost would range from $91.9 million to $128.3 million.
section_title: Negotiated Agreement Could Lead to Higher Emissions Reductions at Less Cost
paragraphs: Following EPA’s initial proposal, representatives of the plant owners and environmental groups (Grand Canyon Trust and Environmental Defense Fund) met, at the recommendation of EPA, to discuss the most cost-effective control option. This led EPA, in early 1991, to facilitate discussions between these representatives to find a mutually acceptable control option. According to EPA, its participation included assisting in drafting documents to support a potential agreement between the parties and providing technical assistance. These parties met repeatedly during a 3-month period to discuss control options and their related costs in an attempt to clarify all options and their costs. As a result of these discussions, the parties reached a negotiated agreement to, among other things, reduce sulfur dioxide emissions from the power plant by 90 percent. According to EPA, its final decision, issued in October 1991, substantially adopted the terms of this agreement.
The agreement specified the time frames in which the emission control technology should become operational and also the manner in which it is to be operated. The agreement specified that the three primary pieces of equipment (“scrubber” modules) should become operational over a 3-year period—the first unit by November 1997, the second by November 1998, and the third by August 1999. The emissions from all three units will be subject to a 90-percent emissions reduction that will be averaged on a 365-day plant operation basis to determine compliance. The agreement also specified that the maintenance schedule for the plant would shift so that some planned maintenance would occur in the winter, thereby shutting down some of the plant’s equipment and further reducing wintertime sulfur dioxide emissions.
According to Salt River Project officials, two factors account for the lower expected project costs. First, the agreement allows the power plant to determine its compliance with EPA’s emissions limit on an annual rather than a monthly rolling average basis as was initially proposed. Determining compliance on an annual basis is a less stringent requirement (than determining compliance on a monthly basis) because it gives the plant more days over which it can average the short-term increases in emissions that would occur when one of the scrubbers is malfunctioning or being repaired. As such, the plant operators can still comply with EPA’s emissions limit without installing the expensive backup equipment they would have to otherwise operate on days when the primary equipment is not operating. According to a project engineer for the Salt River Project, with its compliance determined on an annual basis, the plant can operate its emission control equipment most days at a rate greater than that needed to cut emissions by 90 percent to make up for those days on which emissions are not controlled because the equipment is not operating.
Second, the agreement delays the initial installation of emission control equipment by almost 3 years, from January 1995 to November 1997, which allows the plant operators to complete the project in a more cost-effective manner. According to the plant operators, the additional time allows them to, among other things, better plan the engineering. That is, the operators have had more time to study emission control technologies and select what they consider to be the best technology at the lowest cost. Salt River Project officials also told us that staging construction over a longer period would allow them to reduce labor costs as compared to those with an accelerated construction schedule.
Despite this almost 3-year delay, EPA concluded that the terms of the final rule would result in greater visibility improvement than the proposed rule. In fact, EPA estimated that the emissions limit in its final rule would reduce by two-thirds the amount of pollution that would have been allowed under the proposed rule.
section_title: EPA Estimated an Approximately 7 Percent Improvement in Wintertime Visibility
paragraphs: EPA’s estimate of an approximately 7 percent improvement in the winter seasonal average visibility results primarily from significant improvements expected to occur during certain winter weather conditions. Other less substantial improvements are expected on other winter days. EPA initially estimated an approximately 14 percent improvement primarily on the basis of a study by the National Park Service, although the agency revised its estimate to approximately 7 percent to reflect the results of other analyses and studies. EPA noted that its revised estimate may be understated because it does not take into account other visibility improvements (1) below the rim of the Grand Canyon, (2) in seasons other than winter at the Grand Canyon, and (3) year round at other nearby national parks. Appendix II provides additional details on studies of visibility impairment in and around the Grand Canyon.
subsections: section_title: Visibility Improvement Estimates Were Based on Several Studies
paragraphs: EPA’s initial estimate of an approximately 14 percent visibility improvement relied primarily on data from a Park Service study—the National Park Service Report on the Winter Haze Intensive Tracer Experiment (WHITEX)—of visibility impairment in the vicinity of the Grand Canyon. The study was designed to evaluate a variety of modeling approaches to attribute visibility impairment from a single source—NGS. Specifically, various models were to be evaluated for their ability to link NGS’ emissions to winter visibility impairment at the Grand Canyon and other nearby national parks. In conducting this study, researchers released a traceable chemical from NGS’ smokestack and tracked its movement to monitoring stations in the region, including at the Grand Canyon. The study concluded that NGS contributes approximately 40 percent on average to wintertime visibility impairment in the canyon and approximately 60 to 70 percent during the winter weather conditions when NGS has the most severe effect.
After considering information received following its proposed rule, EPA revised its estimate of the winter seasonal average visibility improvement to approximately 7 percent. This estimate translates into an increase in the average visual range from about 124 miles to about 133 miles. In revising its estimate, EPA relied on the WHITEX study, air monitoring information, and a visibility study conducted by the plant owners.
EPA estimated that the largest improvements from reducing emissions from NGS would occur during certain winter weather conditions. These conditions are, according to EPA officials, (1) high relative humidity, which facilitates the conversion of the plant’s gaseous sulfur dioxide emissions to visibility-impairing sulfate particles, and (2) wind patterns that transport the emissions to the Grand Canyon. EPA estimated that these conditions occur between 10 and 15 times per winter, lasting from 3 to 5 days each occurrence.
However, a Park Service official who was a principal investigator on the WHITEX study told us that the effect of emissions from NGS on impaired visibility at the Grand Canyon during these episodes can be mitigated by local weather patterns. The official explained that, due in part to local weather conditions, the most severe effects occur approximately two to three times per winter, lasting from 5 to 7 days each time. This official explained that visibility can be impaired during these winter weather conditions because of both naturally occurring impairment—mist, fog, clouds—and man-made sources, primarily NGS. However, the official noted that photographic and air monitoring data show that the impairment from man-made sources can continue for several days after the naturally occurring conditions have dissipated. In addition, the evidence also indicates that impairment from man-made sources is perceptible even on some days that include natural impairment.
In addition to improvements during certain winter weather conditions, EPA also estimated visibility improvements on other winter days. These estimated improvements were measured in terms of “changes in contrast,” which, like visual range, is another method of measuring visibility improvements. EPA defined “contrast” as the percentage difference between the brightness of a scenic element and its background. Using this method, EPA estimated that reducing NGS’ sulfur dioxide emissions by 90 percent could result in at least a “perceptible” change in visibility conditions (defined as a 4-percent change in contrast) on approximately 100 days during the winter. EPA later dropped these estimates due to an error in the calculations. The plant owners attempted to correct this error and estimated 54 days of at least a perceptible change. Later, using the results of their own visibility study, the owners reduced this estimate to 6 days.
EPA also relied on other studies and analyses in calculating the degree of visibility improvement that could result from reducing NGS’ sulfur dioxide emissions. These studies included a review of the WHITEX study by a committee established by the National Academy of Sciences’ National Research Council and a separate visibility study conducted by the plant owners. After reviewing the techniques and data used in the WHITEX study, the committee concluded that, on some days during the study period, NGS contributes significantly to visibility impairment in the Grand Canyon. However, the committee also concluded that the WHITEX study was not sufficient to make a quantitative determination of the exact fraction of visibility impairment at the Grand Canyon that is attributable to NGS. The power plant owners’ study found a lesser impact on visibility in the canyon. The study estimated that the average wintertime visual range would improve by no more than 2 percent as a result of reducing NGS sulfur dioxide emissions by 90 percent. In reviewing this information, EPA concluded that there was reasonable agreement between the plant owners’ study and the WHITEX study. EPA noted that the major difference is that the WHITEX study led to the conclusion that peak impairment conditions occur more frequently and that nonpeak impairment conditions are greater than zero more often than found during the plant owners’ study.
section_title: EPA Believes Visibility Benefits May Be More Than the 7 Percent Estimated
paragraphs: EPA identified additional benefits from reducing sulfur dioxide emissions by 90 percent that suggest there may be more than a 7-percent improvement in the winter seasonal average visibility. These benefits include a greater visibility improvement that would occur below the rim of the Grand Canyon and improvements during seasons other than winter at the Grand Canyon and year round at other nearby national parks. First, EPA’s estimated 7-percent improvement in the winter seasonal average visibility did not reflect the more pronounced improvement expected below the rim of the canyon because the air below the rim may be more affected by NGS’ emissions. The National Research Council committee’s review of the WHITEX study noted that meteorological evidence, still photographs, and time-lapse video suggested that sulfur concentrations (indicative of plant emissions) in the canyon might have been considerably greater than those that were observed at the monitoring station used during the WHITEX study—a monitoring station located at the rim of the canyon. The Park Service subsequently established an air monitoring station within the canyon, and, from its results, EPA found that visibility impairment was worse in the canyon than was measured at the rim of the canyon. EPA said that it did not quantify the additional visibility improvement expected below the rim of the canyon due to the limited amount of data available and a limited understanding of the air movements below the rim.
Second, EPA’s estimated 7-percent improvement in the winter seasonal average visibility did not reflect benefits in seasons other than winter at the Grand Canyon and throughout the year at other nearby national parks. EPA explained that, on the basis of information received during its public comment period, emissions from NGS may significantly impair visibility year round at the Grand Canyon as well as at other national parks in the region. For example, a study prepared by the Grand Canyon Trust, which modeled emissions from NGS over a 5-year period, indicated visibility impairment at the Grand Canyon in seasons other than winter. Furthermore, the study suggested that the emissions could impair visibility in surrounding national parks between 60 and 80 percent of the time year round. EPA said that the emissions controls, required by the final rule, would significantly reduce if not eliminate NGS’ contribution to visibility impairment in nearby national parks.
section_title: EPA and the Owners Used Contingent Valuation to Estimate the Monetary Value of Visibility Benefits
paragraphs: Both EPA and the plant owners estimated, using contingent valuation, the monetary value of visibility improvements from reducing sulfur dioxide emissions from NGS. EPA estimated annual nationwide values ranging from $90 million to $200 million. The plant owners estimated a nationwide value of $2.3 million. Although relying on the same methodology, the studies were technically different. EPA’s estimate was extracted from existing research because EPA was under a court-ordered deadline to complete the rulemaking. Therefore, it did not have time to conduct original research to estimate the monetary value of visibility improvements at the Grand Canyon National Park. Unlike EPA in its reliance on existing research, the owners specifically designed contingent valuation research to estimate the visibility improvements they expected from emissions controls at the plant. Nonetheless, the owners did not complete their study for several reasons, including time constraints. Instead, they used the pilot study results to estimate an annual nationwide value of the visibility improvements they expected to occur. Neither studies’ results were used as a basis for EPA’s final rule that established an emissions limit because, as a result of the negotiated agreement, project costs dropped below the $100 million threshold requiring such an estimate.
subsections: section_title: EPA Estimated the Monetary Value of Visibility Benefits but Did Not Use the Estimates
paragraphs: EPA set out to estimate the monetary value of visibility improvements to comply with the terms of Executive Order 12291. This order provided that, to the extent permitted by law, agencies should not take regulatory action unless the potential benefits to society outweighed the potential costs to society. The order required agencies, including EPA, to prepare a regulatory impact analysis that included a cost-benefit analysis. Agencies were to do this for proposed rules that, among other things, were likely to result in an annual effect on the economy of at least $100 million. In such cases, an agency’s analysis was required to describe the benefits—expressed in monetary terms, if possible—as well as potential costs. If the analysis did not show that benefits exceeded costs, the agency was to explain any legal reasons why the regulation should still be promulgated.
When EPA first proposed the rule requiring emissions controls at NGS, it believed that the cost-benefit analysis was required, as the annual cost was thought likely to exceed $100 million (estimates ranged from $91.9 million to $128.3 million). By the time EPA issued its final rule, however, the estimated annual cost—as a result of the negotiated agreement—had decreased to $89.6 million. Accordingly, the Office of Management and Budget exempted EPA from the requirements for a regulatory impact analysis, including a cost-benefit analysis.
When EPA began the cost-benefit analysis, it was faced with court-ordered deadlines to complete this rulemaking effort. As a result of the deadlines, EPA effectively had less than 6 months to complete its analysis and did not have time to conduct original research to estimate the monetary value of limiting the plant’s emissions. Instead, EPA estimated the value of limiting these emissions by extrapolating from the results of earlier contingent valuation research that sought to value the benefit of reducing air pollution at national parks across the country, including those in the Southwest. EPA in its proposed rule, estimated that the monetary value of visibility improvements would range from $1.30 to $2.50 annually per U.S. household. Later, to reflect the revised estimate of visibility improvement from approximately 14 percent to approximately 7 percent, EPA decreased its annual household value to $0.75 to $1.75. EPA estimated the monetary value, nationwide, would range from $90 million to $200 million in the year 2000.
The owners also used contingent valuation to estimate the monetary value of visibility improvements in response to EPA’s use of the existing study and monetary value estimate. Unlike EPA in its reliance on existing related research, the owners specifically designed their study to value visibility improvements they expected from emissions controls at the plant. Nonetheless, the owners did not complete their research because they did not see value in doing so and because of time and resource constraints. Therefore, the owners’ estimated value of expected visibility improvements was based on the results of a pilot test of a proposed survey instrument. The owners’ study estimated the national value of visibility benefits to be $2.3 million. This equates to about $0.023 per U.S. household. Appendix III discusses similarities in the two contingent valuation studies and their specific technical differences.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to the Department of the Interior and to the Environmental Protection Agency for their review and comment. In written comments, Interior officials said that they found the report to be generally accurate and a fairly balanced summary of certain technical aspects of EPA’s decision to require emissions reductions at NGS. (See app. IV.)
We received comments from directors of two EPA offices: the Director of the Office of Policy Analysis and Review, representing the Acting Assistant Administrator of the Office of Air and Radiation, and the Director of the Office of Economy and Environment, representing the Assistant Administrator of the Office of Policy, Planning, and Evaluation. EPA’s Office of Air and Radiation said that the report was generally accurate and complete. EPA’s Office of Policy, Planning, and Evaluation raised concerns about our discussion of contingent valuation methodology (see app. I) and our comparison of the contingent valuation studies conducted by EPA and the plant owners (see earlier in this letter and app. III). Both offices also suggested technical clarifications, which we incorporated as appropriate.
Office of Policy, Planning, and Evaluation officials said that appendix I of our report gives undue attention to the guidelines of a blue-ribbon advisory panel convened by the National Oceanic and Atmospheric Administration, which they believe implies that the recommendations have some relation to EPA’s use of contingent valuation. The officials also said that our report does not give a balanced view of contingent valuation and places too much emphasis on arguments critical of contingent valuation. The officials suggested that we include a reference to a specific article by a prominent researcher in support of contingent valuation, a reference to comments EPA has made on using contingent valuation to assess natural resource damages, and arguments to counter the advisory panel’s guidelines regarding surveys and formats used in eliciting information from survey respondents.
As we state in our report, we are not taking a position on the appropriateness of contingent valuation. Appendix I provides a brief overview of the contingent valuation method, including public policy uses, historical development, characteristics of a contingent valuation study, criticisms, and some further issues. In this context, we summarize the advisory panel’s guidelines because we believe that the panel’s deliberations represent valuable critical and impartial thinking related to contingent valuation. The appendix does not evaluate the merits of the advisory panel’s guidelines or of various arguments for or against the use of contingent valuation methodology. However, to make our presentation more complete, we made minor modifications to the text, added a reference to the article recommended by EPA, and expanded our discussion of alternative survey modes. We did not add the other information suggested by EPA because it is beyond the scope of this appendix.
Office of Policy, Planning, and Evaluation officials also said that appendix III of our report does not suitably explain the reasons for differences in EPA’s and the owners’ contingent valuation studies and the appropriate interpretations of these differences. Without such explanation, the officials believe that a reader may erroneously conclude that there is something wrong with the reliability of the method. EPA suggested that we not directly compare the studies because neither was pursued to the point where any useful comparisons could be made, and that we emphasize what EPA believes to be the more important problems with the owners’ study, such as incomplete documentation, questionable statistical techniques, and a sample size that was too small.
As we note in our report, appendix III describes the similarities in the two contingent valuation studies and specific technical differences between them—in their purpose, design, and implementation—which led to their different estimates of nationwide values. Because neither study used sampling strategies that would allow nationwide projections, we question the certainty of both studies’ estimates of nationwide values. Some of the differences in these studies added uncertainty to their estimates of nationwide values. Wherever information was available, we point out the reasons for these differences and the impact they had on both studies’ results; however, in some instances, neither study had sufficient information, and further testing would be needed to determine the effects of each difference on the estimates. It was not our intent to complete or refine either study to provide a valid nationwide projection, but merely to point out how each study was conducted and why they produced different results. Nevertheless, to clarify that the studies were done separately, we made minor revisions to the text of this letter.
section_title: Scope and Methodology
paragraphs: To obtain information for this report, we reviewed EPA’s documents on its NGS regulatory action. The information included numerous analyses of the plant’s effects on visibility at the Grand Canyon and the visibility improvements that might be expected from the addition of emissions controls. The information also included analyses on the economic costs and benefits of emissions controls. We supplemented this information through discussions with officials of various federal agencies: EPA, the Department of the Interior and its Bureau of Reclamation and National Park Service, the National Oceanic and Atmospheric Administration, and the Department of Energy’s Western Area Power Administration. We reviewed and compared two contingent valuation studies that estimated the monetary value of expected visibility improvements from emissions controls. One of the studies was conducted by RCG/Hagler, Bailly, Inc., and was the basis for EPA’s estimates. The other was conducted for the NGS owners by Decision Focus, Incorporated. We also interviewed officials of the Salt River Project; Decision Focus, Incorporated; RCG/Hagler, Bailly, Inc.; the Navajo Nation; the Environmental Defense Fund; the Grand Canyon Trust; the Grand Canyon Visibility Transport Commission; Air Resource Specialists, Inc.; Northern Arizona University; and others. To describe the contingent valuation methodology, we searched and reviewed economic literature.
We conducted our review from January through December 1997 in accordance with generally accepted government auditing standards. While we did not independently verify or test the reliability of data provided by the agencies or the plant owners, EPA used this information in reaching its regulatory decision.
As arranged with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after the date of this letter. At that time, we will send copies to the Ranking Minority Member of the Committee; the Administrator, EPA, the Secretary of the Interior, and other interested parties. We will also make copies available to others on request. If you or your staff have any questions, please call me at (202) 512-3841. Major contributors to this report are listed in appendix V.
section_title: Contingent Valuation Methodology
paragraphs: The contingent valuation method uses surveys to ask respondents for information that can be used to provide estimates of how much they—and often, by extension, society—are willing to pay for a certain program or policy, such as those designed to improve the quality of some environmental or natural resource amenity. Proponents of contingent valuation methodology believe that it is a valuable technique for making inferences about these values, particularly in cases in which consumer behavior is not (easily) observed. However, the use of the contingent valuation method has been the subject of controversy, particularly in applications involving non-use values.
This appendix provides a brief overview of the contingent valuation method. The first section describes the public policy uses of the method and aspects of its historical development. The second section describes characteristics of a contingent valuation study and presents the suggestions intended to improve contingent valuation practice made by a blue-ribbon panel of social scientists. The third section discusses some of the criticisms that have been leveled at the contingent valuation method. The final section discusses some further issues related to the use of contingent valuation, including some aspects related to its application to regulatory proceedings.
subsections: section_title: The Contingent Valuation Method and Its Development
paragraphs: Contingent valuation studies use surveys to elicit information about how much people would be willing to pay for particular goods or services. These values can be important in estimating the benefits applicable to a wide variety of public policy contexts, including those that require regulatory or environmental impact analyses. While in many instances, economic benefits can be estimated using information on market prices and quantities—because under certain conditions price and quantity data can be used to estimate underlying values held by consumers—in other cases, often involving natural resources or environmental goods, complete market information may not be available. This could be because markets do not exist at all, as in the case of public goods, or because consumers combine their time with purchases in markets for complementary goods needed to undertake a recreational experience, for example. If the values people do have for these goods are not considered in policy decisions, then less desirable resource management outcomes may occur.
As a general proposition, asking people a question about how much they value a particular item seems a direct way of getting estimates of their value for it. However, economists have generally been skeptical of this approach and have historically viewed market-based methods, or so-called “revealed preference” methods in which actual spending decisions can be observed, as inherently superior to “stated preference” methods. Nevertheless, there are many instances in which no behavioral patterns exist through which consumers reveal the values they hold. In such instances, the contingent valuation method can be thought of as a valuation exercise in which a “contingent,” or hypothetical, market is described for the purpose of replicating the consumer choice framework that is used to generate values for traditional market goods. That is, the approach attempts to create a market-based choice context for goods without (complete) markets, such as public or quasi-public goods, so that through their choices people will reveal their preferences much as they do when making actual spending decisions.
Contingent valuation practice developed using theory and practice from different disciplines, especially economics and survey research. A prominent resource economist, Ciriacy-Wantrup, is generally credited with the suggestion of asking people directly for the values they placed on natural resource programs with public good aspects. The first practitioner of what is now known as contingent valuation was Robert K. Davis who used questionnaires as one way to estimate values people placed on recreational experiences in Maine.
The theory and practice of contingent valuation continued to develop in the 1960s and 1970s, and most of the first applications were to resource and environmental issues. During this period, many contingent valuation studies also examined underlying research issues. Some of this research worked toward grounding contingent valuation within the economic theory of consumer behavior. For example, economic theory includes many well understood theoretical relationships involving a consumer’s utility, income, expenditures, and the conditions under which the concept of willingness to pay is an appropriate measure of underlying value. Also, advances in cognitive psychology contributed to understanding the possible biases in a respondent’s answers that may result from such things as the choice of wording or order of questions. Furthermore, researchers gained practical experience in designing, implementing, and analyzing contingent valuation studies. By the 1990s, researchers had performed hundreds of such studies.
The federal government sponsored many of these studies, as various federal agencies performed and funded contingent valuation studies and general research on contingent valuation. These included the U.S. Army Corps of Engineers, the Department of the Interior, and the Environmental Protection Agency (EPA). EPA in particular was interested in the analytical potential of contingent valuation in a variety of environmental regulatory contexts in light of Executive Order 12291 (and its successor), which required executive branch agencies to more systematically examine the costs and benefits of certain of their proposed regulations. In EPA’s case, this involved the use of contingent valuation to estimate the benefits associated with various pollution control regulations.
Although contingent valuation is a methodology that can be used for different purposes, it has become inextricably linked with the measurement of non-use values. Interest in non-use values has been heightened in part because of the possibility that they may be considered in resource damage assessment contexts. The federal government in its role as trustee may include non-use values when calculating damages to be recovered through litigation. The Comprehensive Environmental Response, Compensation and Liability Act of 1980 (CERCLA), or Superfund, provided government officials the right to sue on behalf of the public for resource damages resulting from release of hazardous materials. The Congress directed the President, who delegated the responsibility to the Department of the Interior, to develop regulations applicable to resource damage assessment. After a number of groups challenged the regulations, a federal appeals court upheld Interior’s adoption of contingent valuation methodology for assessing damages to natural resources and directed Interior to revise its rule to avoid limiting the role of non-use values or “non-consumptive” values in the calculation of damages. The grounding of the Exxon Valdez led to the passage of the Oil Pollution Act of 1990, which required the Department of Commerce, acting through the National Oceanic and Atmospheric Administration (NOAA), to develop regulations governing damage assessment. The Exxon Company USA could be subject to liability under the provisions of the Oil Pollution Act and sponsored research concerning contingent valuation, much of which is critical of the ability of contingent valuation to measure non-use values accurately.
section_title: Key Characteristics of Contingent Valuation Studies
paragraphs: In an overview of contingent valuation practice, a leading resource economist stated that while there is no “standard approach,” contingent valuation studies typically include three general features. First, a contingent valuation study contains descriptions of the policy or program at issue and the likely environmental effects so that respondents can understand the good they are valuing. Second, a contingent valuation study contains a framework or mechanism for eliciting willingness to pay. Several mechanisms have been used in contingent valuation studies, such as open-ended questions (How much would you be willing to pay?), payment cards (Select an amount from a list of options.), and referendum formats (Would you vote for the described proposal if your taxes increase by $10?). Third, a contingent valuation study may gather information on socioeconomic variables and attitudes about the environment. This information can be used to estimate willingness-to-pay functions using econometric techniques.
Researchers have developed many methods to implement contingent valuation studies within this broad framework. Additionally, within the context of the method’s development, there have been analytical debates over the merits of particular aspects of contingent valuation practice. The Exxon-sponsored research represented a change in the discussion of contingent valuation issues in that much of this research was carried out by economists and others who were not primarily specialists in natural resource and environmental issues. These researchers raised some new issues and provided new emphasis on other issues on which there had been ongoing analytical debate.
As part of the process by which it developed its regulations related to oil spill damages, NOAA convened a blue-ribbon advisory panel to address a variety of issues, including the fundamental question of whether the contingent valuation method was capable of providing reliable estimates of non-use values for use in resource damage assessments. The panel’s report stated that contingent valuation “can produce estimates reliable enough to be the starting point of a judicial process of damage assessment, including lost passive-use (non-use) values.” Although NOAA was concerned with the use of contingent valuation in the damage assessment context, the NOAA guidelines have applicability to the contingent valuation method more generally. We refer to them because we believe that the NOAA panel’s deliberations represent valuable critical and impartial thinking related to improving the use of contingent valuation. The panel listed some guidelines for producing credible studies and noted some strong concerns about the results of some contingent valuation studies that it reviewed. Although its conclusion gave credence to the views of those who favor the use of the contingent valuation method, adherence to the panel’s suggestions would likely require changes in contingent valuation practice in that none of the studies the panel reviewed had been carried out to its suggested standards.
The panel’s report listed a number of suggestions for producing high-quality contingent valuation studies. Some of these suggestions pertained to the importance of the underlying survey research in contingent valuation studies, in which the survey instruments often have to provide a substantial amount of background material in a manner that is accessible to the respondents. The panel suggested (1) using probability sampling and appropriate statistical sampling procedures, (2) subjecting the survey instruments to pretesting, and (3) taking steps to reduce nonresponse rates. Additionally, the panel suggested that contingent valuation studies disclose information on the sample selection process and provide information on survey instruments and responses. The panel stated a strong preference for the use of in-person surveys as superior to telephone or mail surveys. The panel’s report stated that it is “unlikely that reliable estimates of values could be elicited with mail surveys.” The panel also suggested that it was desirable to pretest any photographs that would be used to convey information to respondents.
In terms of the elicitation format, the panel suggested that the referendum format, as opposed to open-ended elicitation, was desirable. In its basic form, a referendum format contingent valuation study describes a proposal to provide a specific improvement in an environmental good, and the survey respondents are asked if they would support this proposal as if it were a referendum item to be voted on. As part of the proposal, a “payment vehicle” is described, such as a tax increase or a utility bill increase, and each respondent is given a specific per person (or per household) dollar amount that this proposal will cost. The voting question is a dichotomous choice (“yes” or “no”), and, in conjunction with other information gathered in the survey, such as environmental attitudes, income level, etc., econometric techniques appropriate to dichotomous choice situations can be used to determine a measure of willingness to pay for the described proposal on the basis of the observed pattern of yes or no votes. Supporters of the referendum model argue that it creates a contingent market mechanism with which consumers are familiar. First, consumers are familiar with “posted price” market choice contexts.Second, the referendum format itself is familiar to people as a method of expressing political preferences.
The panel was concerned that steps be taken so that results of contingent valuation studies conform to common notions of economic rationality. The NOAA panel endorsed the use of follow-up questions asking respondents the reasons that they voted the way they did as well as questions designed to test how well the respondent understood the program or policy at hand. The panel also suggested that survey respondents be provided with a reminder that paying for the non-use good at issue would result in a smaller budget to spend on other goods and services and that they be told of any available substitutes.
One aspect of rationality is that, generally speaking, people are willing to pay more for greater amounts of a good. In its deliberations, the panel had concerns about evidence presented in one contingent valuation study that estimated willingness to pay “for the cleanup of all lakes in Ontario was only slightly more than willingness to pay for cleaning up lakes in just one region” and in another study that estimated “willingness to pay to take measures to prevent 2,000 migratory birds (not endangered species) from dying in oil-filled ponds was as great as that for preventing 20,000 or 200,000 birds from dying.” The panel suggested that a contingent valuation study demonstrate its sensitivity to these so-called “scope effects.”
section_title: Criticisms of Contingent Valuation Method Have Been Raised
paragraphs: Some economists and other analysts have voiced criticisms of contingent valuation methods. An overarching concern among some observers is that contingent valuation does not adequately capture true estimates of willingness to pay. One component of this criticism is that respondents make choices but that these choices do not require real economic commitments. Also, particularly with respect to non-use values, critics argue that it can be difficult for respondents to comprehend a particular environmental or resource valuation issue, or to distinguish what researchers envision as a well-defined specific issue from a more general “warm glow” effect. Furthermore, some critics argue that the statistical estimation process by which willingness-to-pay estimates are produced from survey responses can be imprecise. At the same time, proponents of contingent valuation have made arguments that respond to many of these criticisms.
One criticism of the contingent valuation method is that contingent markets do not create choice contexts with binding budget constraints and the financial consequences associated with “real” choice contexts. In general, the issue is that by actually spending a certain amount of money, an individual or household can no longer spend that money on something else. Thus, the goods and services that are purchased presumably represent the true preferences of the individual or household. In contrast, responding yes to a contingent valuation question does not financially bind the respondent in the same way. Proponents of the contingent valuation method have been also concerned with this issue and suggest that appropriate steps in survey design serve to reduce the problem. Others maintain that the existence of opportunities for strategic misrepresentation, among other problems, reduces the usefulness of the contingent valuation method. “. . . wired differently than the economic model of fully formed, stable, rational preferences requires. While the consumer’s wiring may produce patterns of market behavior that will often be approximated well by the economist’s model, when we approach the consumer from a different angle, asking direct and unusual questions about values, we find alarming variations from the standard economist’s story. All these consumers, so normal and rational on the outside, are revealed to be shells filled with vast rule-books of heuristics written by natural selection. Throw these people a curve ball, in the form of a valuation question that fails to fit a standard heuristic for market response, and the essential mindlessness of the organism is revealed.”
Critics have also argued that estimates produced by contingent valuation studies may not be limited to values of the specific environmental amenity under consideration but may also incorporate a variety of broader values. The NOAA panel recognized the concern that contingent valuation estimates may contain a “warm glow” component associated with supporting worthy causes.
One additional criticism is that resulting estimates of willingness to pay can be particularly sensitive to the statistical methods used. One analyst examined a variety of statistical issues in contingent valuation estimation and concluded that the estimates were sensitive to context effects, including anchoring effects, as well as the choice as to how statistical outliers were handled.
Proponents of contingent valuation have responded to many of the arguments developed by critics. In particular, a prominent contingent valuation researcher has written an overview article that provides many general arguments in favor of contingent valuation, as well as a point-by-point discussion of several specific issues raised by critics of contingent valuation.
section_title: Further Considerations
paragraphs: Many observers believe that the use of contingent valuation seems likely to continue to grow. Some aspects of the contingent valuation method that are not entirely analytical may also influence the future path of its use in regulatory and damage assessment proceedings. One aspect concerns potential problems with incorporating evolving scientific understanding of the specific environmental issues crucial to a given policy evaluation into survey instruments that take time to develop, implement, and analyze. Another aspect involves consideration of the geographic extent of the affected population. A further issue concerns the “calibration” of willingness-to-pay estimates for use in regulatory or damage assessment proceedings. Additionally, some practitioners of contingent valuation are concerned that some of the specific recommendations of the NOAA panel may inappropriately preclude other analytical alternatives that may prove to be superior or more cost-effective.
Federal regulatory actions often trigger specific requirements and may involve deadlines. For instance, the National Environmental Policy Act (NEPA) of 1969 requires federal agencies to prepare an environmental impact statement if a proposed federal action is likely to significantly affect environmental quality. Although neither NEPA nor its implementing regulations require non-use values to be considered, non-use values have been considered in NEPA proceedings. A contingent valuation study requires an accurate description of the likely change in environmental amenity, which in turn requires careful consideration of the underlying environmental impacts, perhaps including anthropological, atmospheric, biological, and physical components. In some contexts, much of the underlying scientific information may have to be developed during the environmental impact statement process. Because there are many steps required to develop, implement, and analyze survey instruments, there is a chance that the willingness-to-pay estimates will be produced on the basis of descriptions of expected environmental impacts that do not accurately reflect later scientific understanding, or that regulatory decisionmaking time frames are lengthened as that information is incorporated. In other regulatory contexts, such as the one involving the Navajo Generating Station (NGS), court-imposed deadlines may influence not only a decision to undertake a contingent valuation study, but decisions as to how underlying scientific understanding is incorporated into the survey research process. If a particular policy action is controversial or disputed, the accuracy of the underlying description of environmental impacts is likely to be challenged as leading to inaccurate calculations of willingness to pay for those improvements.
Much of the analytical discussion focuses on estimates of per person or per household willingness to pay, and how sensitive or robust such estimates may be to particular choices in underlying description or analytical technique. However, for use in benefit-cost analysis or in estimating damage assessments, the issue of how many people are affected—for instance, how many people are assumed to have non-use values—is important in calculating gross benefit numbers. For contingent valuation estimates of recreation values, samples of recreationists offer a fairly straightforward way of defining the relevant population. For non-use values, the choice of the relevant population may not be so clear. For resources of national significance, researchers may reasonably consider that the national population is the relevant population and may design a study on the basis of that premise. In other cases, the answer is less clear. In any event, it is possible to generate a large benefit number when even fairly small estimates of willingness to pay are multiplied by 100 million, approximately the number of households in the country.
Some observers have argued that contingent valuation estimates of willingness to pay need to be adjusted, or calibrated, because of the inherent limitations. In its deliberations, the NOAA panel reported that it was “persuaded that hypothetical markets tend to overstate willingness to pay for private as well as public goods” and that the same bias would be likely to occur in contingent valuation studies. In its proposed rule, the Department of Commerce (NOAA) recommended a 50-percent calibration factor to adjust for biases of unknown magnitude but of an upward direction. Although a comparison of contingent valuation estimates with other estimates is not possible for non-use values, some researchers have more recently compared contingent valuation estimates with “revealed preference” estimates in a number of studies for which both kinds of estimates were produced. The researchers examined a variety of recreation studies and also cases in which amenities might be capitalized into an asset price, such as a price premium a house with a beautiful view might command over a similar house without the view. The authors located 83 studies that provided 616 comparisons of contingent valuation to revealed preference estimates. The authors reported that contingent valuation estimates were “smaller, but not grossly smaller, than their  counterparts.” Although some contingent valuation estimates were larger than their counterparts, the authors concluded that suggestions for a routine downward adjustment of contingent valuation estimates appear unwarranted.
Some advocates for the use of the contingent valuation method have voiced concern over some of the NOAA panel’s suggestions. In particular, the panel’s strong preference for in-person surveys over mail surveys has been criticized by proponents of mail surveys, as has the panel’s preference for the referendum format. The panel’s preference for in-person surveys had much to do with the fact that sampling frames available for mailing provide incomplete coverage for the national population. The panel also was concerned that targeted respondents can review the subject of the questionnaire before deciding to respond, so those most interested in the subject may choose to respond. Proponents of mail surveys counter that other survey methods, such as in-person interviews, also have their drawbacks, such as problems caused by the presence of an interviewer, which may bias responses, or pressures on respondents to answer quickly while the interviewer is present. They also add that mail surveys of large samples offer significant cost savings over in-person interviews.
The panel’s preference for the referendum format was based on a number of factors, including the fact that people are “rarely asked or required in the course of their everyday lives to place a dollar value on a particular public good.” Even though open-ended elicitation is not familiar, some researchers point to results from experimental economics indicating that posted price choice contexts perform poorly relative to open-ended contexts in “early rounds” of bidding situations in which respondents are not experienced. Given that respondents are not likely to be well informed in many contingent valuation contexts (at least for non-use goods), these researchers argue that the experimental results that people overpay in early rounds suggests that a “one-round” referendum may lead to an overstated willingness to pay. Other research suggests that the specific price that a referendum survey respondent is confronted with—the bid price—may lead to anchoring effects, so the resulting willingness-to-pay estimates may be too high. In contrast to the typical practice in which bid prices are distributed randomly to respondents, this research suggests that some initial investigation incorporating open-ended valuations could be useful in avoiding the assignment of high bid prices to respondents with low values (and vice versa).
section_title: Studies of Visibility Impairment Causes in and Around Grand Canyon National Park
paragraphs: The largest benefit EPA expected to occur at the Grand Canyon National Park as a result of reducing emissions from NGS was an improvement in visibility during certain winter weather conditions. These conditions are expected to occur approximately 10 to 21 days each winter. EPA initially estimated that reducing the sulfur dioxide emissions by 90 percent would improve the winter seasonal average visibility by approximately 14 percent. These estimated improvements were based in part on a study of visibility impairment in the vicinity of the Grand Canyon by the National Park Service (NPS). EPA revised its estimate to approximately 7 percent after considering information from other studies that suggested that NGS has a lesser effect. EPA noted that its revised estimate may be understated because of other unquantified visibility improvements.
subsections: section_title: EPA Initially Estimated an Approximately 14 Percent Improvement in Visibility
paragraphs: The Clean Air Act requires that, upon a finding that it is reasonable to anticipate that an emissions source may be causing or contributing to the impairment of visibility in certain national parks or wilderness areas, the relevant state or EPA is required to determine an emissions limit for the source that reflects the best available retrofit technology (BART). In determining an emissions limit, EPA is required to take into consideration, among other things, the costs of reducing emissions and the degree of improvement in visibility that may reasonably be anticipated to result from the use of such technology. In September 1989, EPA proposed to attribute visibility impairment in the Grand Canyon to emissions from NGS and, as a result, was required to carry out a technology assessment of NGS.
EPA’s determination of an emissions limit relied on data from an NPS study (the National Park Service Report on the Winter Haze Intensive Tracer Experiment [WHITEX]) of visibility impairment in the vicinity of the Grand Canyon. WHITEX was designed to evaluate the ability of using a variety of modeling approaches to attribute visibility impairment from a single source, NGS. Specifically, various models were to be evaluated in their ability to link NGS’ emissions to visibility impairment at the Grand Canyon and other nearby national parks. According to WHITEX, wintertime meteorological conditions in the area are characterized by several periods of stagnation in which air pollutants can be trapped by a persistent thermal inversion, resulting in a distinct visible surface haze layer. Although several earlier investigations have been conducted to determine the origins of the haze, WHITEX was a more comprehensive effort to address persistent questions about the nature and sources of the winter haze conditions.
According to EPA and NPS officials, the emissions from NGS can have the largest effect during certain weather conditions that include (1) high relative humidity, which facilitates the conversion of the plant’s gaseous sulfur dioxide emissions to visibility-impairing sulfate particles, and (2) wind patterns that transport the emissions to the Grand Canyon. EPA estimated that these conditions occur between 10 and 15 times per winter, lasting from 3 to 5 days each occurrence. However, NPS officials explained that the effect of emissions from NGS on visibility impairment at the Grand Canyon during these conditions can be mitigated by local weather patterns. According to these officials, due in part to local weather conditions, the most severe effects occur approximately two to three times per winter, lasting from 5 to 7 days each time. These officials explained that visibility can be impaired during these winter weather conditions because of naturally occurring impairment—mist, fog, clouds—and because of man-made sources, primarily NGS. However, the officials noted that photographic and air monitoring data show that visibility impairment from man-made sources can continue for several days after the naturally occurring conditions have dissipated. In addition, the evidence also indicates that impairment from man-made sources is perceptible even on some days that include natural impairment.
WHITEX, carried out during January and February of 1987, relied on injecting a unique chemical into NGS’ smokestack and tracking this chemical to air monitoring stations that were placed around the region, including at the Grand Canyon. The study concluded that NGS was the single largest contributor to visibility impairment in the Grand Canyon during the days for which air monitoring data were available. WHITEX’s results indicated that, for days on which air monitoring data were available, NGS contributed approximately 40 percent on the average to wintertime visibility impairment and approximately 60 percent to 70 percent during the worst visibility impairment conditions.
EPA’s estimates of the degree of visibility improvement used WHITEX data to establish the relationship between NGS’ emissions and visibility impairment in the canyon. EPA explained that, because of the complex terrain in and around the Grand Canyon, the WHITEX data provided a more reliable estimate than the models often used to estimate improvements in visibility. Specifically, EPA used the ratio of sulfur dioxide emissions at NGS to sulfate particles in the Grand Canyon attributable to NGS. Using this ratio, EPA applied a “linear rollback” model, which used regression analysis techniques to estimate the level of visibility impairment that would result from a given level of NGS’ sulfur dioxide emissions. The model’s formula contained terms that attempted to account for, among other things, the percentage of sulfate that contributes to overall visibility impairment, the percentage of NGS’ contribution to total sulfates, and the removal rate of the control technology.
EPA’s analysis of NGS’ emissions reductions and the resulting visibility improvements was complicated in several ways. For example, EPA had to determine whether to account for the possibility that a linear relationship may not exist between NGS’ sulfur dioxide emissions and the resulting visibility impairment in the Grand Canyon. In other words, would a reduction in NGS’ sulfur dioxide emissions result in a proportional or less-than-proportional reduction in visibility impairment in the canyon attributable to NGS? EPA explained that WHITEX showed that the conversion of sulfur dioxide to visibility-impairing sulfate particles is greater in a moisture-rich environment (e.g., clouds or fog) and the lack of such an environment tends to limit such conversion. However, EPA also explained that other studies showed that moisture-rich environments may also inhibit the conversion of sulfur dioxide to visibility-impairing sulfate particles because the compounds with which the sulfur dioxide might react (typically hydrogen peroxide and ozone) may combine first with other compounds and lessen the conversion of sulfur dioxide to the visibility-impairing sulfate particles. EPA determined that this issue was insignificant because adequate quantities of compounds, such as hydrogen peroxide, would likely exist during the winter and other studies of trends in various parts of the country did not indicate any significant nonlinearity. However, EPA did modify its model to address another complication. This complication stemmed from the possibility that reducing sulfur dioxide emissions could increase the amount of other visibility-impairing compounds to be formed. EPA was concerned that, if sulfur dioxide was reduced, ammonia that would have combined with sulfur dioxide to form visibility-impairing sulfate would instead combine with nitrogen oxides, forming ammonium nitrate. EPA modified its model to account for this potential “nonlinear” complication.
EPA assessed a variety of different scenarios to determine the potential visibility improvements. First, the model assessed the potential visibility improvements (in terms of visual range—expressed in kilometers) under average conditions found during the WHITEX study period that could result from emission control rates of 70, 80, and 90 percent at NGS. Second, the model assessed the potential visibility improvements under the worst-case conditions found during the WHITEX study period that could result from removal rates of 70, 80, and 90 percent. EPA assessed scenarios that assumed a linear relationship between sulfur dioxide emissions and visibility impairment and other scenarios that assumed a nonlinear relationship.
Model results showed a dramatic increase in estimated visibility improvements during the worst-case conditions compared to improvements during the average conditions. For example, the results showed visibility improvements under average conditions that ranged from approximately 11 percent for a 70-percent level of emissions reduction to approximately 14 percent for a 90-percent level of emissions reduction. These figures compare to the model’s estimates of visibility improvements under worst-case conditions that ranged from approximately 60 percent for a 70-percent level of emissions reduction to approximately 94 percent for a 90-percent level of emissions reduction.
Because modeling average conditions does not necessarily represent actual conditions on a given day, EPA also examined potential visibility improvements using actual data collected during the WHITEX study. In cases where total visibility impairment data were not available (because of weather conditions during the study period—i.e., during periods of cloud cover), EPA reconstructed data that were measured during the study period. This analysis found visibility improvements that ranged, on average, from approximately 23 percent to 43 percent, depending on the level of emissions reduction. This level of visibility improvement was approximately 2 to 3 times higher than the estimated improvement found using average visibility conditions and approximately half the values found under the average worst-case conditions.
section_title: EPA Estimated Improvements on Other Winter Days
paragraphs: In addition to improvements during certain winter weather conditions, EPA also estimated visibility improvements on other winter days. These estimated improvements were reported in terms of “changes in contrast,” which, like visual range, is another method of measuring visibility. EPA defined “contrast” as the percentage difference between the brightness of a scenic element and its background. With this method, EPA estimated that, using information developed in the WHITEX study and extrapolating it to the winter period (and applying a nonlinearity factor), reducing the emissions from NGS by 90 percent would have the following effects on visibility: (1) at least a “perceptible” change in visibility conditions (defined by EPA as a 4-percent change in contrast) approximately 100 days of the total winter days, (2) a “quite noticeable” change in visibility conditions (10-percent change in contrast) approximately 58 days of the total winter days, and (3) a “very apparent” change in visibility conditions (20-percent change in contrast) approximately 21 days of the total winter days.
Although these estimates illustrate the varying effect of NGS on visibility during the winter, EPA eventually dropped the estimates. EPA explained that its calculations were in error because it did not take into account natural atmospheric scattering of light. Similar calculations were made by the plant owners, who attempted to correct for EPA’s error, and also showed that differing levels of improvements can be expected during the winter months. The plant owners’ estimates showed that 54 days, rather than EPA’s estimate of 100 days, would have at least a perceptible change. Using the results of their own visibility study, the plant owners argued that reducing the emissions by 90 percent would result in (1) approximately 4 days during the winter of a perceptible improvement in visibility, (2) approximately 2 days during the winter of a quite noticeable improvement in visibility, and (3) 0 days during the winter of a very apparent improvement in visibility.
section_title: National Academy of Sciences Reviewed WHITEX Study
paragraphs: The National Academy of Sciences’ National Research Council established a committee to evaluate the WHITEX study. The Council noted that one of the study’s greatest weaknesses was that no measurements of visibility impairment were made below the rim of the Grand Canyon, within the canyon itself. The Council noted that meteorological evidence, still photographs, and time-lapse video suggest that sulfur concentrations in the canyon might have been considerably greater than was observed at the monitoring station used during the study and located at the rim of the canyon. On the basis of the data presented in the WHITEX study, the Council concluded that, on some days during the study period, NGS contributed significantly to haze in the Grand Canyon. However, the review also concluded that the study was not sufficient to ascertain the quantitative contribution by NGS to haze at any given time. The authors of the WHITEX study agreed with the Council that two of the quantitative techniques used in the study could not be used alone or to exactly apportion NGS sulfur dioxide emissions to visibility impairment at the canyon. Rather, the authors explained that they used these quantitative analytical techniques in conjunction with qualitative techniques to make reasonable estimates of NGS’ effect.
section_title: Plant Owners Conducted Separate Visibility Study
paragraphs: Concerned with what they believed to be shortcomings of the WHITEX study, the plant owners conducted their own visibility study. This study was similar to the WHITEX study in its use of a unique tracer through NGS’ smokestack and air quality monitoring stations around the Grand Canyon. The owners’ study found that NGS’ sulfur dioxide emissions contributed less to visibility impairment at the canyon than the WHITEX study concluded. The owners’ study estimated that a 90-percent reduction in NGS’ sulfur dioxide emissions would not improve the average visual range in winter by more than 2 percent. In reviewing the owners’ study, EPA concluded that there was reasonable agreement between the findings of this study and the findings of the WHITEX study with respect to NGS’ peak contribution to sulfate and visibility impairment at the Grand Canyon. According to EPA, the major difference was that the WHITEX study led to a conclusion that the peak visibility impairment conditions occur more frequently and that the nonpeak visibility impairment conditions are greater than zero more often than found by the plant owners’ study.
section_title: In-Canyon Visibility Impairment Was Estimated but Not Quantified
paragraphs: NPS established an air monitoring station within the canyon that addressed one of the shortcomings of its study that was noted by the National Research Council—that impairment measurements within the canyon were not made. Preliminary results from the monitoring station showed that visibility impairment in the canyon was worse than the impairment measured at the monitoring station used during the WHITEX study, which was located at the rim of the canyon. Data from the new monitoring site below the rim of the canyon confirmed that air transport and conversion processes below the rim of the canyon are sometimes decoupled from those processes above the rim. EPA also explained that photographic data taken during the WHITEX study indicated that airflow below the rim of the canyon could result in higher visibility impairment due to the trapping of pollution. EPA said that it did not quantify the expected visibility improvements below the rim of the canyon due to the limited amount of data available and a limited understanding of the air transport mechanisms below the rim of the canyon.
section_title: EPA Revised Its Estimate to an Approximately 7 Percent Improvement in Visibility
paragraphs: Following a public comment period, EPA revised its estimate that reducing NGS’ sulfur dioxide emissions by 90 percent could improve the winter seasonal average visibility above the rim of the canyon from its initial estimate of approximately 14 percent to approximately 7 percent. In revising its estimate, EPA relied on the two visibility studies and other air monitoring information. EPA noted that it still believed that the primary improvement in visibility would stem from reductions of emissions from NGS during winter weather conditions. However, EPA also noted that other visibility improvements will occur, including improvements below the rim of the canyon, during seasons other than winter at the canyon, and at other national parks in the area. Therefore, EPA noted that its estimate of approximately 7 percent is likely an underestimate.
EPA explained several factors that tend to make the estimate an understatement. First, EPA’s estimate did not include the more pronounced improvement that would be realized in the canyon, below the rim. EPA noted that, a comparative analysis, prepared by Air Resource Specialists, Inc., of 3 years (1988 to 1991) of sulfate levels from above-rim and in-canyon air monitoring stations, showed in-canyon visibility impairment up to 10 times greater than that measured on the rim and concluded that high sulfate conditions below the rim typically last from 3 to 5 days longer than do those observed at the rim. In addition, the study concluded that there is a high degree of confidence that NGS is responsible for at least 90 percent of the visibility impairment at the Grand Canyon during these periods.
Second, EPA noted that the principal improvement will likely be during certain wintertime weather conditions. EPA’s approximately 7 percent estimate reflects an average over the 5-month period from November through March. Since the winter weather conditions during which NGS can have its largest effect occur intermittently throughout the 5-month period, EPA expects that the visibility improvement during these winter weather conditions is substantially greater than 7 percent.
Third, EPA did not estimate the expected visibility benefits to be realized during nonwinter seasons at the Grand Canyon or at other surrounding national parks. NGS is located near several national parks located on the Colorado Plateau—which in addition to the Grand Canyon include Arches, Bryce Canyon, Canyonlands, Capitol Reef, Mesa Verde, and Zion. Two studies were submitted to EPA that estimated NGS’ year-round impacts on these other parks. One study, by the Air Resource Specialists on in-canyon visibility impairment, estimated the year-round effects of NGS. The study estimated the visibility effects of NGS’ emissions for every hour from December 1985 to November 1990 and concluded that NGS’ emissions were present at the Grand Canyon (1) an average of 35 percent of the time in the winter and (2) near or above an average of 20 percent of the time 8 months of the year. The study concluded that when NGS’ emissions are not in the Grand Canyon, they are most likely affecting another national park in the area. The study estimated that NGS’ emissions are present in these other parks on average at least 50 percent of each month throughout the year. The other study, prepared by Latimer and Associates, analyzed the impact of NGS’ emissions on impairment during all seasons in these national parks (including the Grand Canyon) for the same 5-year period. The study concluded that haze impacts generally are highest in the Grand Canyon in the winter and calculated that perceptible sulfate haze impacts due to NGS’ emissions occured in all other parks and in each season during the 5-year period modeled. The study concluded that since NGS is surrounded by national parks, the likelihood is high that at least one park is impacted at any given time.
section_title: Contingent Valuation Studies Used to Value Visibility Improvements at the Grand Canyon National Park
paragraphs: This appendix discusses two contingent valuation studies related to EPA’s 1991 regulatory action that established an emissions limit for NGS. One study was performed for EPA, the other for the plant owners. Both studies set out to value changes in visibility at the Grand Canyon, had survey instruments that were carefully designed by their researchers, and showed that people were willing to pay some amount to improve visibility at the Grand Canyon. The studies were different, however, in the specifics of what they were to value and how they went about doing so. Some of these differences added uncertainty to the studies’ results. It is less clear how the other differences affected results, and testing would be needed to determine the effects due solely to each of these differences.
subsections: section_title: Why EPA and the Plant Owners Estimated a Monetary Value of Visibility Improvements
paragraphs: EPA set out to estimate the monetary value of visibility improvements in order to comply with Executive Order 12291. The order provided that, to the extent permitted by law, agencies should not take regulatory action unless the potential benefits to society outweighed potential costs to society. The order required agencies, including EPA, to prepare a regulatory impact analysis that included a cost-benefit analysis. Agencies were to do this for proposed rules that, among other things, were likely to result in an annual effect on the economy of at least $100 million. In such cases, an agency’s analysis was required to describe the benefits—expressed in monetary terms, if possible—as well as the potential costs. If the analysis did not show that benefits exceeded costs, the agency was to explain any legal reason why the regulation should still be promulgated.
Cost-benefit analyses were expected to conform to guidelines developed by the Office of Management and Budget and EPA. The guidelines allowed EPA considerable flexibility in estimating its benefits. They stated, among other things, that the scope and precision of analysis should depend on the specific requirements of authorizing legislation, the quality of underlying data, the scientific understanding of the problems to be addressed through the regulation, and resource constraints at EPA.
EPA, according to officials, was faced with such resource constraints. It had, in effect, a court-ordered deadline for completing its estimate of the monetary value of visibility improvements expected from limiting the plant’s emissions. On the basis of a 1982 lawsuit filed by environmental groups and a subsequent settlement agreement and revisions to the settlement agreement between EPA and these groups, EPA was under court order at this time to determine whether a specific pollution source caused or contributed to the visibility impairment at the Grand Canyon and, if so, issue a finding to that effect by August 31, 1989. In addition, following any finding, EPA was to conduct a best available retrofit technology (BART) analysis on the identified source. And, if the analysis indicated emissions controls would improve visibility at the Grand Canyon National Park, EPA was to propose regulations requiring their installation and use in order to achieve the emissions limit representing BART. Under the court order, EPA was to complete its technology analysis by February 1, 1990. This was less than 6 months from August 31, 1989, when EPA was required to issue its finding as to whether NGS was a source of impairment.
EPA concluded that it did not have time to complete original research to estimate a monetary value of the specific visibility improvements expected from emissions controls at the plant. The agency chose, instead, to extract the monetary value from the results of existing contingent valuation research related to visibility changes at the Grand Canyon. EPA’s decision to estimate benefits based on contingent valuation was, according to a former EPA official who was the project economist for this rulemaking, partially an attempt to foster a wider review of the use of the contingent valuation methodology so that, if accepted, it could be used on other environmental policy issues and regulatory decisions.
The existing study, “Preservation Values for Visibility Protection at the National Parks,” was partially funded by EPA through a cooperative agreement with the University of Colorado Center for Economic Analysis and performed by the research firm of RCG/Hagler, Bailly, Inc. The existing study was designed to advance the state of the art in estimation of use and non-use values because existing methods were considered to be quite limited when the need for such values was increasing for reasons including Executive Order 12291 requirements. The researchers were Lauraine G. Chestnut and Robert D. Rowe (EPA researchers). EPA selected this study, from among others, because it included many recent methodological developments intended to respond to earlier criticisms of the contingent valuation methodology for valuing visibility conditions. EPA also selected this study because its estimates of the monetary value of visibility improvements were conservative when compared with another earlier study’s.
The plant owners also used contingent valuation to estimate a monetary value for the visibility benefits expected from limiting the plant’s emissions. In response to EPA’s use of the existing study and monetary value estimate, the owners decided to conduct their own study and contracted with a research firm, Decision Focus, Incorporated, to do so.
section_title: The Studies Valued Visibility Improvements
paragraphs: Both the study EPA used and the owners’ study set out to value visibility improvements at the Grand Canyon National Park. The studies, however, valued different degrees of visibility improvement. That is, the study on which EPA based its estimated monetary value was intended to value a much broader visibility issue than the wintertime visibility improvements expected from emissions controls at the plant. The owners’ study, on the other hand, set out to value wintertime visibility improvements the owners expected would result from controlling their plant’s emissions.
EPA expected, as stated in its proposed rule for a 90-percent emissions limit, that there would be an approximately 14 percent improvement in the winter seasonal average visibility at the Grand Canyon. The improvement was expected to occur over a 30-year period, which was EPA’s estimate for the remaining useful life of the plant. However, the study on which EPA based its estimate valued changes in annual average visibility that would last forever at several individual national parks, including the Grand Canyon. The photographs used in the study, which survey respondents were asked to value, were labeled summer days. And the broader study valued different visual range improvements than EPA expected would occur from limiting emissions: a 61-percent improvement in visual range; a 29-percent improvement in visual range; and a 26-percent degradation in visual range.
The owners’ contingent valuation study was specifically designed to measure the wintertime visibility improvements expected from emissions controls at the plant. This study asked respondents to value five different scenarios of visibility improvements. Interviewers described, for respondents, the expected visibility improvements of each scenario and showed them photographs that illustrated the improvements.
The five scenarios, shown in table III.1, were chosen by Decision Focus to relate to potential regulatory actions by EPA. The last three represent the different types of winter improvements Decision Focus hypothesized would result from emissions controls at the plant.
section_title: Survey Instrument Development
paragraphs: We found that the researchers for both the study EPA used and the owners’ study made very thorough efforts in developing their survey instruments. They followed accepted survey research standards to ensure the validity of their survey instruments. As a result of these efforts, the survey instruments should have measured the visibility concepts the researchers intended them to measure and with wording that the researchers found to be most effective for their studies’ purposes. And, as one would expect because the studies were intended to measure different visibility improvements, the survey instruments provided respondents different information about what they were to value and used different photographs to demonstrate visibility improvements.
Careful survey design, in our view, is critical to averting problems with bias or comprehension. It is needed because people, on whom these researchers relied to value visibility changes, are complex and their reactions to specific words or concepts are not always predictable. If the right questions are not asked or if questions are not asked in the right way, researchers are less likely to obtain high-quality results. Asking the right questions in the right way is both science and art. It is a science because it is guided by empirical evidence and uses many scientific principles developed from various fields of applied psychology, sociology, cognitive research, and evaluation research. It is an art because it requires anticipating the respondents’ interactions with the survey instrument.
Both research groups pretested their survey instruments to avoid bias and comprehension problems. Pretesting involves administering survey questions to people who represent the population to be surveyed. This can be discussions with focus groups or in-person or telephone interviews and is intended to identify problems that researchers can correct before administering their survey instrument to a larger group. The researchers for the study EPA used held two rounds of pilot tests, each involving about 10 respondents. Then, after revising the survey instrument, the researchers had it peer reviewed by sociologists familiar with issues concerning national park visitors and survey design issues, economists familiar with contingent valuation, and an atmospheric scientist familiar with visibility. They then hired professional interviewers to conduct a final pretest with 20 respondents. The owners’ researchers held two rounds of focus groups to explore basic assumptions about visibility improvements, each followed by a round of telephone interviews. Then, after analyzing the information gathered, they conducted two more rounds of focus groups and preliminary in-person interviews. Drawing on the information gathered, the researchers then developed a survey instrument that they revised following two more rounds of focus groups, two rounds of test interviews, and finally a pretest with 22 respondents that was conducted by professional interviewers.
subsections: section_title: Contextual Information and the Use of Photographs
paragraphs: For contingent valuation surveys to elicit useful information about respondents’ willingness to pay for specific environmental improvements, we believe researchers must ensure that the respondents understand exactly what they are being asked to value. In deciding what kind and how much information to provide respondents, researchers must weigh providing enough and properly ordered information with the possibility of overloading respondents or being criticized for trying to lead respondents.
The survey instruments, for both the study EPA used and the owners’ study, provided respondents different background information and used different photographs to depict changes in visibility. The obvious reason for these differences, in our view, is that EPA used existing research designed to value different visibility changes than those expected because of emissions controls at the plant. Therefore, agency researchers could not have been expected to describe the environmental improvement expected from emissions controls at the plant. Another important reason is that contextual information and photographs are matters of researcher choice and are an area where contingent valuation may be more art than science.
subsections: section_title: Contextual Information
paragraphs: The nature of the information the research groups provided respondents was different. For example, in the owners’ study, before respondents were asked to value five levels of visibility improvements, they were given specific background information. Among other things, the respondents were told that on high visibility days, one can see more than 100 miles at the Grand Canyon; the rural southwest has some of the clearest air in the country; the actual amount of pollution at the Grand Canyon National Park is very low compared with the amount in cities; most visitors come in the summer period; and if any of the programs to improve visibility that the respondents were asked to value were implemented, certain older power plants, already meeting all current state and national air pollution standards, would have to install and maintain new equipment to remove pollutants.
These background statements, in our view, might have caused respondents to minimize their concern over visibility problems at the Grand Canyon and accordingly to assign lower willingness-to-pay values for visibility improvements. On the other hand, these statements could be exactly what the respondents needed in order to understand what they were to value.
Another example of contextual information is from the study EPA used. In that case, prior to asking respondents to value visibility changes, the researchers first introduced respondents to several nonvisibility effects of air pollution at national parks. Before asking the respondents to value visibility changes, the researchers first asked them to prioritize nonvisibility effects that were happening or could happen in national parks due to people’s activities outside park boundaries, for example, injury to vegetation and historic structures from air pollution. Then, later in the questionnaire, following the valuation questions, respondents were asked to separate, from their willingness-to-pay values, any amount they had included for nonvisibility improvements.
A possible effect of introducing these additional results of air pollution before the valuation questions, in our view, might have been that respondents assigned higher willingness-to-pay values than they might otherwise have. And subsequent efforts to separate out any inflated amounts might not have been successful. On the other hand, introducing nonvisibility issues might have been a critical step in ensuring that respondents valued only visibility improvements, that by identifying these other, nonvisibility, effects, respondents might have been better able to value only visibility changes.
Another aspect of contextual information is its level of detail. The study EPA used and the owners’ study were very different in terms of their level of detail. The owners’ researchers, knowing they were to value visibility improvements from emissions controls at the plant, were able to give respondents very specific descriptions of visibility conditions and ask very specific questions about the visibility improvements. In contrast, EPA’s researchers provided respondents more general descriptions of visibility conditions and possible events that might change them.
This difference in detail, we believe, could cause substantial variation in the values the studies’ respective respondents placed on visibility improvements, depending upon the cognitive patterns of the respondents. A test, administering the two survey instruments to randomly selected samples from the same population, would be needed to determine the effects due solely to the level of detail in the survey instruments.
section_title: The Use of Photographs
paragraphs: The researchers for both the study EPA used and the owners’ study used photographs to illustrate the visibility improvements respondents were to value. The characteristics of the photographs they chose (e.g., size or season) were different. The effect, if any, these differences had on the values respondents assigned to visibility improvements is not known. Testing would be needed to determine the effects due solely to the differences in photographs.
The selection of photographs, according to a former official who was EPA’s project economist for this rulemaking, demonstrates a challenge in accurately depicting what it is respondents are to value. At a minimum, according to this official, problems stemming from the selection of photographs can increase the uncertainty of the results and provide another avenue for criticism of results. At worst, problems can yield biased results with an unknown direction of bias.
Both research groups said they selected their photographs to minimize bias. EPA’s researchers used NPS photographs that represented four visibility conditions on summertime days: 15 percent of the summertime days (the best conditions), 20 percent, 40 percent, and 25 percent (the poorest conditions). The photographs were taken at the same time each day and selected to minimize variations in extraneous factors such as clouds and snow. While the owners’ researchers also used NPS pictures and selected them with the assistance of a leading visibility scientist from NPS, the pictures they chose were very different. The owners’ researchers selected photographs representing different seasons and weather categories, for example, summer clear skies, winter clear skies, winter overcast, and winter layered haze events. These photographs were also selected to represent issues, including the change in the Grand Canyon’s appearance from season to season (primarily due to the change in the sun’s angle), summer afternoon thunderstorms’ tendency to frequently obscure views, and the impact the time of day has on the appearance of vistas.
In addition to different weather patterns, the researchers used different numbers, sizes, and qualities of photographs. The EPA researchers used four photographs printed to be mailed to respondents—each picture was 3 by 5 inches. The owners’ researchers used 12 pictures, measuring 8 by 12 inches, mounted on display boards to be shown to respondents during in-person interviews. We believe these differences could cause substantial variation in the values respondents assigned for visibility improvements. A test showing the different photographs to two randomly selected samples, from the same population, would be needed to determine the effects due solely to the differences in photographs.
section_title: Survey Administration
paragraphs: EPA’s and the owners’ researchers administered their survey instruments in different ways. EPA’s researchers used mail questionnaires to contact the 710 respondents in its study, while the owners’ researchers contacted 202 respondents in-person. There are trade-offs when choosing between these two survey techniques. Both techniques have their strengths and weaknesses.
In-person interviews, in our view, have both strengths and weaknesses. Strengths of in-person interviews include researchers’ being able to control the amount of information respondents have available when answering specific valuation questions and interviewers’ ensuring that survey questions are asked in the exact order the researchers intended. Because in-person interviews are conducted in one setting, respondents are less likely to be interrupted by outside events, for example, personal or family illness, that might change their perspective when answering questions. Furthermore, these interviews are also generally more successful with respondents whose reading levels are low in comparison to the complexity of the questions. The weaknesses of the method include higher costs because interviewers not only must be trained, but they must also travel to and from interviews—some of which may not be successful. In addition, interviewers, by their presence, may affect how respondents answer questions. For example, respondents may provide an answer they believe the interviewer wants or give any answer just to further the interview process.
Strengths of mail questionnaires include being substantially cheaper than in-person interviews. Being less expensive, mail questionnaires can be sent to larger samples than may be possible with in-person interviews and, as such, may be more appropriate for research issues requiring nationwide results, such as the valuation of visibility improvements at the Grand Canyon National Park. Mail questionnaires allow respondents time to carefully consider each question and their response. Weaknesses of the method include the possibility of respondents’ having more information than researchers intend them to have when they answer a specific question (because they can skip back and forth between questions or read ahead). Also, when mail questionnaires are used, there is no one who can assess for researchers whether respondents understand the questions or what it is they are to value. While some in the research community tend to prefer in-person interviews for contingent valuation surveys, mail questionnaires have not been proven to be a less valid technique for collecting data.
section_title: Willingness-To-Pay Results and How They Were Calculated
paragraphs: While both studies showed respondents were willing to pay some amount for visibility improvements at the Grand Canyon, the researchers used different techniques to calculate their willingness-to-pay values. EPA, in extrapolating from the results of the earlier contingent valuation study, made various assumptions and judgments about how to translate values elicited for larger benefits to values for the narrower, specific benefits of this case. And the plant’s owners, in calculating results from their pilot study, used a data-trimming technique that removed a fixed amount of data from the calculations. These techniques added uncertainty and possible bias to the study results. Additional uncertainty may have also been added through adjustments both groups of researchers made in response to the final results of visibility studies that showed less visibility improvement than the studies valued. Any uncertainty in these results was magnified when both groups of researchers projected their willingness-to-pay values to the nation without conducting national samples. EPA then projected its nationwide results to the number of years it expected the regulation to be in effect.
subsections: section_title: EPA’s Extrapolation Process and Results
paragraphs: EPA’s goal was to identify that portion of the broader study’s willingness-to-pay values relevant to the expected visibility improvements from emissions controls at the plant. To accomplish this, EPA’s researchers did the following: created a database of the results from the original research that were pertinent to visibility at the Grand Canyon. The original research used six surveys, three of which contained questions about visibility at the Grand Canyon National Park. The results of the three survey instruments were what researchers combined for the database. determined, for the database, the relationship between visibility improvements and the willingness to pay for the improvements. The original study made three willingness-to-pay estimates—one each for a 61-percent visibility improvement, a 29-percent visibility improvement, and a 26-percent degradation of conditions. The researchers calculated a mean willingness-to-pay value for each of these levels of improvement. determined for each level of improvement, using regression analysis, the relationship between the individual willingness-to-pay values and other factors such as respondents’ age, household income, gender, and history of visiting national parks, which were important in determining the willingness to pay. using this empirical relationship, predicted willingness-to-pay values for the approximately 14 percent visual range improvement that EPA initially expected would occur from the addition of emissions controls at the plant. using both sensitivity analyses and comparisons to related past contingent valuation studies, tested the validity of their predicted willingness-to-pay values.
EPA’s researchers’ estimate of the monetary value of visibility improvements, expected from a 90-percent emissions reduction, ranged from $1.30 to $2.50 per year per U.S. household. EPA recognized that extrapolation, by definition, added considerable uncertainty to the resulting values. Nevertheless, EPA believed that the results were sufficient to serve as indicators of the direction (i.e., negative or positive) and the order of magnitude (i.e., whether the values were in millions, tens of millions, or hundreds of millions) of the values.
section_title: Owners’ Data-Trimming Practice and Results
paragraphs: The owners’ researchers calculated the mean willingness-to-pay values for households for each of the five visibility programs included in their study. They calculated these values using a data-trimming procedure that involved removing a fixed amount of data (first the highest and lowest 5 percent of the willingness-to-pay values and then the highest and lowest 10 percent of the willingness-to-pay values) from both ends of the data distribution curve and then calculating “trimmed means.” According to the researchers, they used trimmed means because the ordinary means of the untrimmed data were grossly distorted by a very small number of outliers. According to the researchers, the ordinary mean is the correct statistic under traditional welfare economic theory, if one is willing to ignore distribution consequences, that is, accept a program in which—in the worse case—all the benefits are accrued by one individual. Trimming, according to the researchers, is an alternative that avoids this extreme case and results in willingness-to-pay value being based on a more central part of the distribution. EPA’s project economist for this rulemaking told us that data trimming in this case was problematic because the distribution of the study’s results was highly skewed, with 90 percent of the willingness-to-pay values being $0.
Data trimming eliminated some results from respondents who said that they would pay a large amount for visibility improvements and some results from respondents who said that they would pay nothing for visibility improvements. This practice greatly affected the owners’ results. For example, for visibility improvements on 20 winter days, the untrimmed mean of the willingness-to-pay distribution was $2.38. This is compared with $0.50 for the 5-percent trimmed mean and $0.02 for the 10-percent trimmed mean. Table III.2 shows these results for each of the five visibility programs the owners’ research examined.
section_title: Additional Uncertainty From Adjustments
paragraphs: Both EPA’s and the owners’ researchers made additional adjustments to their willingness-to-pay results to reflect the final results of visibility studies—results that were not available at the time they began their studies. These adjustments may have added additional uncertainty to the final results.
EPA, as previously discussed, was operating under a court-ordered deadline and began its analysis using the best available, rather than final, estimates of expected visibility improvements. So when the final results became available and were significantly less than the preliminary results its researchers had used to estimate the value of visibility improvements (a winter seasonal average visibility improvement of approximately 7 percent instead of the approximately 14 percent used in the analysis), EPA scaled its willingness-to-pay estimates downward. And while the EPA researchers (at EPA’s request) had designed the computational formulas so that results could be revised when final visibility improvement estimates became available, they also recognized the possibility of adding uncertainty to the results. The researchers said that uncertainties could be added to results if willingness-to-pay values drop off dramatically at some point when visibility changes from summer to winter or from most days to some days. However, they also said that while there was no evidence of such a dramatic drop in mean values, available evidence on the question is quite limited. The revised willingness-to-pay estimates ranged from $0.75 to $1.75 annually per household.
The owners’ researchers, faced with the same time constraints, also were required to begin their research while awaiting final estimates of the visibility improvements expected from emissions controls. Then, after their visibility research efforts were completed, the owner-funded visibility study concluded that visibility improvements from emissions controls would be less than any of the scenarios valued. The study concluded that with a 90-percent emissions reduction, visibility would improve (at least perceptibly) on 6 days. Therefore, the researchers extrapolated their monetary value from the values they had estimated for higher degrees of visibility improvement. The researchers’ final report did not explicitly state a willingness-to-pay value per U.S. household. Rather, the report indicated a total public value of $2.3 million—this equates to approximately $0.023 per household. A senior associate at Decision Focus, Incorporated, was unable to provide specific details on how the final calculations were made because of the time that had passed since the study was completed.
section_title: Uncertainty Is Magnified When Results Are Projected
paragraphs: Any uncertainty in willingness-to-pay values is magnified, in our view, when the results are used to project a nationwide value and applied to the entire period of time to be affected by the regulatory action. While neither group of researchers had a nationwide sample, both projected their results to the nation as a whole. EPA additionally projected its results to the entire time period the regulation would be in effect.
subsections: section_title: Sampling Strategy
paragraphs: Neither EPA’s nor the owners’ researchers had sampling strategies that allow nationwide results. Faced with resource constraints, EPA sacrificed its ability to obtain nationally representative values by choosing to extrapolate from existing research. And while the plant’s owners planned to have nationwide results, time and resource constraints contributed to their not completing the planned study.
The contingent valuation study from which EPA extrapolated surveyed a sample drawn from residents of five states: Arizona, California, Missouri, New York, and Virginia. These states were selected so that there would be variation in the distances between respondents’ residences and the national parks studied. In addition, Arizona, California, and Virginia were selected because they were states with national parks being studied. For each of the selected states, a survey instrument was mailed to respondents whose names were selected from national databases drawn from drivers’ licenses, car and voter registrations, and other sources. Although different surveys were sent to the different states, with different questions about the different national parks, EPA extrapolated from the 710 responses that pertained to the Grand Canyon National Park.
The EPA researchers agreed that their sample did not technically allow a reliable assessment of the U.S. population. However, they believed that they had compensated for the partial sample by adjustments they made to the willingness-to-pay estimates. These adjustments were intended to account for socioeconomic differences (e.g., household income, age, sex, and distance of residence from the parks in question) between their sample and the U.S. population. Nonetheless, the researchers also said that they were unable to calculate the expected level of error in the sample.
The plant owners’ sample was drawn from households in two counties—San Diego and St. Louis. This sample, however, was for a pilot study that tested a survey instrument. With appropriate revisions, the survey was to be administered to a much larger sample from which national results could be drawn. The owners’ researchers selected the two counties to provide different settings. Then, to select households for interview, the researchers first randomly selected blocks within the counties. And second, beginning with a random predesignated starting point and proceeding in a predetermined manner, they went from household to household until they had conducted five interviews with heads of households that met established age and gender quotas. The researchers set these quotas to ensure that men and young people were interviewed since women and older people are easier for researchers to locate. In total, 202 persons were interviewed.
The owners’ researchers recognized that their sample was not a national sample and planned to conduct a national sample. However, according to a senior associate at Decision Focus, by the time they completed the pilot, there was not enough time remaining to complete and summarize a national sample so it could be used in EPA’s decision-making. In addition to time and resource constraints, according to an official of the Salt River Project—part owner and the operator of the plant—the owners did not authorize the remaining research because they did not see value in doing so.
section_title: Survey Results Were Projected
paragraphs: While both EPA and the owners projected their per U.S. household monetary value to the nation as a whole, EPA also projected these results to the number of years the regulation would be in effect (the number of years the plant was expected to operate). EPA estimated that the monetary value of the visibility benefits would range from $90 million to $200 million in 2000 (measured in 1992 dollars). EPA estimated the present value of the monetary benefit stream, as of January 1992 (expressed in 1992 dollars and discounted using a 10-percent real rate), at $523 million to $970 million.
section_title: Comments From the Department of the Interior
section_title: Major Contributors to This Report
paragraphs: Jonathan T. Bachman Stephen M. Brown Daniel J. Feehan Sue E. Naiberk Cheryl L. Pilatzke Cynthia S. Rasmussen Victor S. Rezendes Pamela K. Tumler The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.
U.S. General Accounting Office P.O. Box 37050 Washington, DC  20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000  or by using fax number (202) 512-6061, or TDD (202) 512-2537.
Each day, GAO issues a list of newly available reports and testimony.  To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone.  A recorded menu will provide information on how to obtain these lists.