From: Lydia Wegman/RTP/USEPA/US

To: "Frey, Nathan J." <Nathan_J._Frey@omb.eop.gov>, "Lee, Amanda I."

<Amanda_I._Lee@omb.eop.gov>
Cc: Joseph

- J Dougherty/DC/USEPA/US@EPA, Erika Sasser/RTP/USEPA/US@EPA, Vicki

Sandiford/RTP/USEPA/US@EPA, John Hannon/DC/USEPA/US@EPA

Date: 12/12/2012 05:08 PM Subject: Secondary RTC

-- Preference Studies section

Nathan and Amanda,

Attached is the section of our draft Response to Comments document that address the visibility preference studies.Interagency Review Draft

– Do Note Quote or Cite December 12, 2012

PM NAAQS Response to Comments

— Section IV.5 (DRAFT)

2 5. Specific Comments on Visibility Preference Studies

The evidence relied upon by the Administrator in determining an appropriate degree of public welfare protection from visibility impairment is predominately derived from a small

number of urban

- focused visibility preference studies. As a result, many comments were received on the adequacy and appropriateness of this limited dataset serving as a basis for the Administrator’s judgments regarding what is requisite national protection for vi sibility impairme nt. Both CASAC and the EPA staff, as well as a few commenters, concluded that while limited, these studies did form a sufficient basis on which the Administrator could make a decision. A majority of commenters, however, expressed the view that the existing preference studies provide an insufficient basis for selection by the Administrator of an appropriate level of public welfare visibility protection for a national standard. These commenters highlighted a

number of limitations and uncertainties associated with these studies as support for this view. Many of the important and fundamental comments are also discussed in the final rule , with further details provided below . As in the final rule, this section organizes and discusses the preference study comments under four broad topic areas, including: (a) limitations and uncertainties associated with the visibility preference studies; (b) preference study methods and design; (c) use of preference study results for determining adversity; (d) the appropriateness of using regionally varying preference study results to select a single level for a national standard.

a. Comments on Preference Study Limitations and Uncertainties

A large and diverse number of limitations and uncertainties associated with the visibility pref erence studies ha s been identified and discussed in the public comments. Many of these same limitations and uncertainties were also identified and discussed by the EPA in the various documents developed throughout this review such as the Integrated Science Assessment, Visibility Assessment, and Policy Assessment, as well as in the proposed rule (see section VI.b.2 beginning at 77 FR 3897 3 , 38990) and in the final rule (section VI.C.1

. e.i).

(1) Comment: Most commenters expressed concern over the small number of preference studies that are available in this review. In particular some note that these preference studies cover just four locations, only three of which occur in the U.S., that the two studies conducted in Washington, DC were pilot studies, not full pre ference studies, and/or that three of the preference studies were conducted in the West, while only one was conducted in the East, thus providing only limited geographic coverage. Typically these same commenters also noted the small number of survey partic ipants, which totaled 852 across the four urban areas, which they assert are too few and unrepresentative nationally. These commenters thus concluded that there is insufficient information, both geographically and demographically, upon which to select a na tional level of a visibility index for purposes of visibility protection.

Response: The EPA notes that it is well aware of the limited nature of the information, which it has described in great detail in the Integrated Science Assessment, Visibility

Asses sment, and Policy Assessment, as well as in section VI.B.2 of the proposed rule (i.e., beginning at 77 FR 3897 3 , 38990). The EPA further notes however that limited information does not preclude the Administrator from making judgments based on the

best avai lable science, taking into account the existing uncertainties and limitationsInteragency Review Draft

– Do Note Quote or Cite December 12, 2012

PM NAAQS Response to Comments

— Section IV.5 (DRAFT)

3 associated with that available science. Thus, in reaching judgments based on the science, the Administrator appropriately weighs the associated uncertainties. The CASAC supported this view and concluded that the available information provided a sufficient basis on which the Administrator could form a judgment about requisite PM

- related public welfare visibility protection. Specifically, CASAC stated “[t]he 20

- 30 deciview range of levels chosen by EPA staff as ‘Candidate Protection Levels’ is adequately supported by the evidence presented. (Samet, 2010b, p. iii). As discussed in the proposed rule (77 FR 38990), the Administrator recognize d and explicitly t ook into account the

uncer tainties and limitations in the science in determining an appropriate degree of protection when she propos ed a level at the upper end of the recommended range.

(2) Comment: Some commenters also made the point that the EPA relied on much of this

same evid ence to reach the conclusion in 2006 that the information was too limited to allow selection of a national standard. For example, API stated:

[T]he bulk of the VAQ preference studies were available during the previous PM NAAQS review and were considered b y the Agency in its establishment of the 2006 PM secondary NAAQS.... The Proposed Rule does not mention this fact and does not explain why many of these same studies now compel EPA to propose this new secondary NAAQS....The Proposed Rule notes in passing that, since the last review of the PM NAAQS, ‘limited information that has become available regarding the characterization of public preferences in urban areas has provided some new perspectives on the usefulness of this information in informing the selection o f target levels of urban visibility protection.’ 77 Fed. Reg. at 38969/2. It is a serious oversight that the Proposed Rule makes no attempt to explain what that information is or how it affects the interpretation of the VAQ preference studies. This ‘limite d information’ is an apparent reference to information provided by Dr. Anne Smith. (API, p. 37)

Response: The EPA agrees that 3 of the four study areas examined in this review were available in the previous review (see discussion in section V I .A.1 of the proposal which specifically identifies the studies from Denver, Phoenix and British Columbia (77 FR 38967/2) as being considered in the last review). However, the EPA disagrees that it has not explained what information

, both preexisting and new, it relied upon in this review to inform its views and provide the basis for its proposal, beginning in section V I .A.3 of the proposed rule (77 FR 38969). In particular, in addition to the substantial recent PM urban air quality information and analyses new to this review, there are three other noteworthy sources of information that have specifically “provided some new perspectives on the usefulness of” the preference studies “in informing the selection of target levels of urban visibility protection.” They include: (1) results from additional urban visibility preference study experiments conducted for Washington, DC by Smith and Howell (2009) which added to the preference data for that location, provided proof that the 2001 study could be replicated, and shed light o n the role of location in preference responses (77 FR 38974,Interagency Review Draft

– Do Note Quote or Cite December 12, 2012

PM NAAQS Response to Comments

— Section IV.5 (DRAFT)

4 38976) 1 ; (2) an extensive review and reanalysis of the urban visibility public preference studies from the four urban areas, including the newly available Smith and Howell (2009) experiments whic h examined the similarities and differences between the studies and evaluated the potential significance of those differences on the study results (77 FR 38893, 389975

- 76) 2 ; and (3) additional analyses conducted in response to questions raised by the CASAC , most importantly a logit analysis presented in the Visibility Assessment (see Appendix J and discussion in Chapter 2) which showed that each city’s responses represented unique and statistically different curves (77 FR 38975

- 76). 3 Taken together, these sources contributed to an increase in the EPA’s knowledge and understanding of the robustness of each survey study ’s results, the appropriateness of comparing each study’s results to the others, and the key uncertainties relevant to data interpretation. In addition, in the last review the decision to not adopt a distinct secondary standard was remanded as contrary to law and failing to provide a reasoned explanation for the decision. As such it is not appropriate for purposes of comparison with the

Administ rator's judgment and reasoning in this review.

(3) Comment: Other commenters stated support for using the preference studies, concluding they provide an adequate basis, in spite of their limited nature. In particular, AMC et al.

state:

We believe that these s tudies provide sufficient results to inform setting a national visibility standard. While the number of studies is small

, they do incorporate spatial variation and, in the case of Denver and Phoenix, varied populations....
EPA should have confidence, rather than uncertainty, in the fact that these studies used different methods and respondents and yield a range of 20

– 24 dv, with one outlier of 29. (pp. 6

- 7)

Response: The EPA agrees that these studies provide a limited but sufficient basis to inform the Administrator’s judgments regarding an appropriate level of public welfare visibility protection (see also response to (1) above). However, the EPA does not agree

that the Washington, D.C. results represent an outlier but instead represent a reasonable 50% acceptability value for that type of urban scene and thus are appropriately included in the range identified for the Administrator to consider.

(1) Comment: A number of othe r technical features of the studies were identified by commenters which they assert limit the usefulness of these studies in representing real world urban views and in eliciting appropriate preferences for urban visibility. For

1 Smith, AE; Howell S (2009). An assessment of the robustness of visual air quality preference study results. CRA International. Washington, DC. http://yosemite.epa.gov/sab/sabproduct.nsf/B55911DF9796E5E385257592006FB737/$File/CRA +VAQ+Pref+Robustness+Study+3+30+09+final.pdf

. 2 Stratus Consulting, 2009. Review of Urban Visibility Public Preference Studies: Final Report.

EPA

- HQ

- OAR

- 2007

- 0492

- 0042.

3 Deck L., Lawson M. 2010. Statistical Analysis of Existing Urban Visibility Preference Studies.

EPA

- HQ

- OAR

- 2007

- 0492

- 0089.Interagency Review Draft

– Do Note Quote or Cite December 12, 2012

PM NAAQS Response to Comments

— Section IV.5 (DRAFT)

5 example, some commenters st ate that the assumption of light extinction at 550 nm is unsupported, technically flawed and invalid for a number of reasons, including that it does not account for the mix of pollutants, which varies substantially within and between urban areas, or how ch anges in that mix affect light extinction at different wavelengths of light, and it misses the role of sky discoloration from NO 2 and carbonaceous particles (i.e., Brown Cloud phenomenon) which can be significant in urban areas and influence human percepti on of urban visibility impairment. Other commenters note that pollutant concentrations in urban areas are not uniform in all directions over the length of a line of sight and that visibility depends upon the site

- path integrated concentration of fine PM. Studies which used photographic data capture the integrated site path. They further state that momentary glances at 2

- dimentional pictures cannot represent the actual way that visibility is perceived outdoors.

Response: First, the EPA disagrees that its choice of measuring light extinction at 550 nm wavelength is ‘unsupported, technically flawed and invalid’. The EPA’s Integrated Science Assessment makes clear that 550 nm represents the “ most sensitive portion of the

spe ctrum for human vision” and has thus traditionally “been used to characterize light extinction and its components” ( U.S. EPA, 2009a , p. 9

- 2)

. The ISA further states that

“ the convention for visibility monitoring purposes is to make measurement at or near 5 50 nm, which is the wavelength of maximum eye response” ( U.S. EPA, 2009a , p. 9

- 8)

. The EPA therefore concludes that it remains logical to select a wavelength of light that is most relevant to human observers when evaluating impacts to the public welfare.

R egarding the comment that 550 nm misses the impacts of NO 2 on sky discoloration, the EPA notes that this PM NAAQS review is intended to only address visibility impairment caused by particles present in the ambient air. Thus, while it is true that sky disco loration

from NO 2 results from its ability to absorb more light in the short wavelength blue portion of the spectrum giving a yellow or brown appearance to the pollution layer or plume, improvement in the Brown Cloud Phenomenon is not the goal of this NAAQ S review and would be expected to occur only indirectly from a secondary PM 2.5 NAAQS for visibility in as much as PM 2.5 controls also reduced co

- emitted NO 2 . The EPA further notes, however, that not every urban or other area suffers from the Brown Cloud

Ph enomenon. At background concentrations NO 2 absorption is generally less than five percent of the light scattering by clean air (Rayleigh scattering), making it imperceptible.” (ISA, p. 9

- 5) Regarding the comment that measuring light extinction using 550 nm “ does not account for the mix of pollutants, which varies substantially within and between urban areas”, the EPA disagrees, noting that the IMPROVE algorithm used to calculate light extinction was designed to specifically take into account the variability in the mix of pollutants from place to place and to measure t he “photopic

weighted absorption efficiency at the 550 nm wavelength” in its calculations of PM

related light extinction. Thus, given the above, the EPA believes it is still appropriate to

use t he 550 nm wavelength in order to measure the maximum impact of ambient PM on light extinction that is perceptible to the human observer.

Second, with respect to the other technical issues raised above, the EPA agrees that photographic and WinHaze images c annot duplicate the outdoor, three dimensional viewing environment. This limitation of the preference study design does introduce someInteragency Review Draft

– Do Note Quote or Cite December 12, 2012

PM NAAQS Response to Comments

— Section IV.5 (DRAFT)

6 degree of uncertainty into the results. However, the EPA does not believe that this uncertainty significantly limits the usefulness of the results. People take pictures of aesthetically pleasing vistas so that they can look at them repeatedly, even though they are not exact replicas of their initial outdoor viewing experience. This shows that they appreciate the photographic view of the scene, commensurate with their appreciation with the scene itself. Thus, the EPA believes photos or digital images can be useful surrogates for the purpose of eliciting public preferences. Regarding the commenters other concern that pollutant concentrations in urban areas are not uniform in all directions over the length of a line of sight and that visibility depends upon the site

- path integrated concentration of fine PM, the EPA notes that both the photos used in two of the preference study ar eas (Denver and B ritish C olumbia ) and the WinHaze images used in two others (Phoenix and Washington, DC) depict visibility impairment over an integrated sight path. The difference between the two methods is that a photo can capture the non

uniformity acros s the sight path while the WinHaze views present a uniform haze integrated across the sight path. Photos of the Denver scene therefore would also have included any layered haze known as the brown cloud that was present at the time that fell

within the inte grated site path

. Despite these variables, the ordering of the results of these studies appear to be explainable based predominantly on scene characteristics

– i.e. people found impairment of the views with the most sensitive scene characteristics (Denve r and Phoenix) more unacceptable at lower deciview values than those with less sensitive scenes (B ritish C olumbia and Washington, DC) regardless of the image depiction methods used. Thus, the EPA concludes that the concerns over study methods and designs identified by commenters are not substantial, and that the preference study results are still sufficiently informative to allow selection of candidate protection levels.

a. Comments on Preference Study Methods and Design

(1) Comment: M any commenters asserted th at the methodologies used in the preference studies are fundamentally flawed. Most of these same commenters also pointed to an assessment of the preference study methodology conducted by Smith and Howell (2009) of Charles River Associates (CRA) as the basi s for their views. These commenters criticized the EPA for not acknowledging that (in their view) the Smith and Howell results undermined the validity of the earlier preference studies. For example:

Smith and Howell (2009) show that VAQ preference study ou tcomes are malleable and depend entirely on the design of the study. Accordingly, such studies do not identify any meaningful threshold of acceptable visibility conditions. Despite Smith and Howell’s conclusions, EPA continues to assert that the VAQ prefer ence studies can be used to identify minimally acceptable visibility conditions even though the Agency has never provided any valid scientific basis for discounting the Smith and Howell (2009) results. (API, p. 38)

Well

- controlled preference studies discus sed by Anne Smith of Charles River Associates at the March 2010 CASAC meeting demonstrated that the judgment of panel members was affected by the order in which photographs were presentedInteragency Review Draft

– Do Note Quote or Cite December 12, 2012

PM NAAQS Response to Comments

— Section IV.5 (DRAFT)

7 and tendency to identify the middle of the range of visibility degre dation as a threshold of acceptability. This points to a potential flaw in these studies and that artifacts caused by these tendencies may have influenced study results. Dismissing these inherent flaws in the existing preference studies and then using

thes e studies to set a secondary NAAQS is arbitrary and capricious. (API, attachment 14, p. 12)

Response: As noted above, the EPA is aware of the issues raised regarding the uncertainties and limitations associated with the preference studies, including thos e associated with study methods and design and specifically solicited comment on how these uncertainties and limitations should be considered (see 77 FR 38990). However, as is explained in the following discussion, the EPA strongly disagrees that the analy sis by Smith and Howell (2009) supports the commenters ’ conclusion that the preference study methodologies and design were fundamentally flawed and instead asserts that the Smith and Howell (2009) experiments increased the EPA’s knowledge and understanding of the findings of the 2001 Washington, DC focus group pilot study (Abt, 2001) in several important ways. Smith and Howell (2009) conducted a series of three sets of experiments based on the 2001 Washington, DC

, focus group pilot study. The first set of e xperiments (Test 1) was designed to replicate the 2001 focus group to explore whether the 2001 study findings could be reproduced. Tests 2 and 3 varied the range and distribution of visibility conditions to examine whether such changes would affect the res ults. As explained below, the EPA asserts that some aspects of the approach used by Smith and Howell (2009) in Tests 2 and 3 are flawed.

It is important to note as an initial matter, that while Smith and Howell (2009) claims to have replicated the Abt 2001 protocol, there were three distinct differences in the methods used by Smith and Howell (2009) and the methods used in the Washington, DC 2001 pilot study. First, the images of alternative levels of visibility degradation in Washington

, DC were prepared a nd presented differently. While both studies used the same original photograph of Washington, DC, the 2001 images showing different levels of visibility degredation were prepared by John Molenar of Air Resource Specialists, Inc. (ARS) using the mainframe c omputer version of WinHaze available at that time. The 2009 Smith and Howell experiments, on the other hand, used images prepared in a more simple manner using the MS

- Windows ©

- based version of WinHaze (Ver. 2.9.0) available by that time from ARS. Second, t he 2001 pilot study used high quality photographic transparencies (“color slides”) prepared by ARS, which were presented on a large format, high quality projection screen. The entire group of participants was seated at the viewing angle and distance from t he projection screen in a darkened room to provide the optimal viewing conditions recommended by John Molenar. The 2009 study used images presented to individuals on a large desktop computer monitor. Third, the method used to recruit participants differed between the 2001 and 2009 studies. While a group of only 9 people can never be a statistically valid representative set of any population, the 2001 focus group participants were recruited by a professional focus group facilitation firm to reflect aspects o f the demographic distribution of the Washington, DC metropolitan area adult population. The recruitment screening method was designed to obtain 9 adults resembling major aspects of the demographic gender, age, residence and incomeInteragency Review Draft

– Do Note Quote or Cite December 12, 2012

PM NAAQS Response to Comments

— Section IV.5 (DRAFT)

8 distribution of the Wash ington, DC area. At least two of the 2001 participants were in each of the following groups: a) Income: at least two people with household incomes under $25k, two with incomes between $25k and $60k, and two with incomes above $60k; b) Age: at least two bet ween the ages 18 and 25, two between 26 and 55, and two over 56; c) Race/Ethnicity: at least two African Americans and two Hispanics; d) Residence location: at least two living in each of Washington, DC, Maryland and Virginia. In contrast, the Smith and Ho well (2009) study did not use a structured recruitments process. Due to the schedule of the Smith and Howell (2009) study, all study participants were employees of CRA, Inc. (where both Smith and Howell are also employed). One variant of Test 1 was conduct ed with participants who work at the CRA Washington, DC office (and who presumably live in the Washington, DC area). The other variant of Test 1 was conducted with participants from the CRA Houston office (and likely live in the Houston area). The same set of images of Washington, DC was shown to both the CRA Washington and Houston participants.

In spite of these differences in study methods, the Test 1 results (Smith and Howell, 2009, p.11) found a “...remarkable degree of replication, given the differences in the samples, and the relatively small sample size of the Abt pilot (nine people).” Furthermore, the preferences for Washington

, DC visibility provided by the Houston participants were effectively the same as both the preferences of the CRA Washington

of fice participants and the Abt 2001 participants.

The EPA identified a number of important implications from the similarity between the Test 1 (2009) and 2001 study results.

1) The change in the presentation methods (i.e., from slide projection to computer monitor, and mainframe WinHaze images versus the simpler Windows WinHaze method) shows no evidence of impacting reported preferences.

2) The replicability of the 50% acceptable c riteria results supports a hypothesis that the 2001 results, although based on a small sample of 9, may be a fair representation of the preferences for the general population.

3) The similarity of the Test 1 Houston results to both the Test 1 Washington

resu lts and the 2001 Washington study suggest that an individual’s preferences for visibility in one location (i.e., Washington, DC) may not depend on whether they live in that location or in another.

As a result of the similarity between the results of these two DC studies and the above implications, the EPA concluded that the Test 1 experiment could legitimately be combined with the 2001 study to effectively more than triple the responses to the original Washington, DC urban visibility preference study, thus increasing the available information from 9 (in the 2001 study) to 31 (with the addition of both the Washington

, DC and Houston CRA participants). Smith and Howell’s Test 1 thus instigated a shift in the EPA’s thinking regarding the usefulness of the avail able Washington, DC preference study data.Interagency Review Draft

– Do Note Quote or Cite December 12, 2012

PM NAAQS Response to Comments

— Section IV.5 (DRAFT)

9 The Smith and Howell (2009) Tests 2 and 3 also provided important information relevant to understanding the results of visibility preference studies. In Test 2 and 3 the range and distribution of visibility condit ions shown to participants were substantially different than the range and distribution shown in Test 1 and the 2001 Washington, DC study. Test 2 removed from the range of Test 1 slides those that showed the 11 worst visibility conditions (all images above 27.1 dv). Test 3 changed the visibility conditions presented in two ways: first, the number of clean or cleaner images were reduced substantially, dropping all images in the lower third of the distribution, and adding two new images of substantially worse visibility (42 and 45 dv), which were substantially above the previous worst scene shown (38.3 dv).

Smith and Howell (2009) found that changing the range of visibility images shown has a pronounced impact on the resulting 50% acceptability level. They con cluded that this is an indication that preferences obtained using the general PS method are not stable, and therefore not an accurate representation of the populations pre

- existing (or “true”) preferences for visibility.

While the Smith and Howell (2009) r esults suggest such a hypothesis, the Test 2 and Test 3 results also are consistent with a very different effect in visibility preference survey results. The range of visibility conditions shown in Test 1 and the original 2001 Washington, DC study were sel ected by EPA to present a good representation of the distribution of visibility conditions expected to occur in Washington, DC with then

current EPA air quality regulations. In other words, the original range and distribution of visibility conditions shown to the participants were near to current conditions, and reflected the “policy relevant” range of expected conditions. This fundamental approach was used in all the existing studies (i.e. Denver, Phoenix, and British Columbia); the range of visibility con ditions reflects current real world conditions. While the participants were not informed that the images they were being shown were essentially then

- current conditions, the range of images would have been consistent with the participants own experiences.

W hen the range of images presented was substantially different, as in Tests 2 and 3, the participants would very likely recognize that the conditions did not depict the real range of conditions. This creates a disconnect between the participants’ own experi ences and expectations and the hypothetical scenario being presented to them through the use of the altered ranges. This creates a form of the well known “scenario rejection bias” that has been observed in many types of stated preference studies. Current m ethods of stated preference research on a range of environmental issues find it is very important to present as realistic and believable a description as possible of the real world conditions matching to participants personal experience and knowledge in or der to produce robust, stable and reproducible results. The effect Smith and Howell (2009) see in their Test 2 and Test 3 methods are likely to be due to just such a scenario rejection biasing influence.

Thus, on the basis of the above discussion, the EPA disagrees that the findings of Tests 2

and 3 in Smith and Howell (2009) make the case that the methodology used in the preference studies is fundamentally flawed and therefore disagrees with commenter

views that the EPA’s use of these studies is somehow in appropriate, scientificallyInteragency Review Draft

– Do Note Quote or Cite December 12, 2012

PM NAAQS Response to Comments

— Section IV.5 (DRAFT)

10 unsound, or arbitrary and capricious.

(2) Comment: Many commenters also asserted that preference study participants were not given guidance on how to interpret the concept of “acceptable urban visibility”. This lack of guidance was considered a detriment by commenters because in their view it renders the responses subjective.

Response: EPA notes that while the overall method used in all the preference studies is consistent (based on the method and elicitation protocol developed by E ly et al., 1991 in the Denver study), each study developed a specific protocol (including the specific language on both background information and the wording of the key questions) for conducting that study’s survey. The Denver and Phoenix studies each gav e the survey participants specific guidance on interpreting “acceptability”.

The Denver study framed the question in the explicit context of setting an urban visibility standard for Denver, and instructed the participants that they were being asked to ide ntify the VAQ that in their opinion should violate an urban visibility standard, and that “...standard violations should be VAQ that is unreasonable/objectionable, and unacceptable visually. Participants were asked to discern ‘how much haze was too much’,

no t to indicate standard violations based on whether haze was detectable unless any amount of haze was more than they could accept” (Ely et al.

, 2001 , p. 6

- 7 )

.

The Phoenix study instructions to participants defined acceptable (or unacceptable) as

“Consider ‘unacceptable’ as visual air quality that is unreasonable or objectionable visibility. Please do not mark a slide ‘unacceptable’ just because you can see some haze, unless you believe that any amount of haze is more than you would tolerate.”

The British Co lumbia study (Pryor et al., 1996) followed the survey protocol in the Denver survey closely, and gave similar guidance to the participants on the meaning of “acceptability” in a standard setting context.

The Washington, DC (2001) study was a focus group de signed to investigate a series of issues involved with conducting group interview type surveys about preferences for urban visibility. A key feature of the 2001 focus group script was to deliberately not give the participants guidance on how to interpret “ acceptable” or “unacceptable” before rating the series of Winhaze images. After the series questions involving rating the images, the focus group moderator lead a discussion about how the participants interpreted the term “acceptable”, and what issues and information they considered in forming their responses. The 2009 series of Washington, DC urban visibility preference experiments followed a similar procedure, although apparently conducted a more abbreviated focus group follow

- on discussion.

Thus, the EPA believes that study participants were given sufficient direction and context so that they could understand the nature of the questions being posed and were all on equal footing in developing subjective judgments regarding “acceptable” and “unacceptable” v isual air quality. Since the goal of these studies was to elicit personal preferences, which by very nature are subjective, the EPA does not view the subjectivityInteragency Review Draft

– Do Note Quote or Cite December 12, 2012

PM NAAQS Response to Comments

— Section IV.5 (DRAFT)

11 of the responses as a weakness of the study design or protocols.

c. Comments on Preference Stud y Results and Adversity

A number of comments were received regarding the EPA’s use of preference study

results to make the determination that adverse PM 2.5 - related visibility effects on the public welfare are occurring. Some of these commenters questioned whether the EPA had made the case that unacceptable levels of visual air quality based on preference study results alone can be equated with an adverse public welfare effect. Other commenters questioned the EPA’s selection of the 50% acceptability criterio n as the appropriate benchmark for determining adversity.

(1) Comment: Several commenters asserted that the EPA had not made a sufficient case for the occurrence of adverse public welfare effects in association with ambient levels of

PM 2.5 :

Thus, EPA seemingly was building the foundation for a determination of what constitutes an adverse effect on visibility in the context of public welfare. However ... EPA subsequently veered toward an oversimplified focus on public acceptance of visibility conditions.... EPA’s discussion of visibility in the Policy Assessment and its proposed rule in the Federal Register focuses entirely on “acceptable” and “unacceptable” visual air quality and make no mention of an “adverse effect” in the context of visibility. EPA’s rel iance on only 3 urban PS represents a paucity of data and a wholesale abandonment of any effort to seek a scientifically measurable adverse effect. (Kennecott Utah Copper LLC, p. 26)

These commenters suggested that unless preference study information is li nked to personal comfort and well

- being or other associated welfare effects, it cannot form the basis of a determination of adversity.

Response : The EPA first notes that the definition of effects on welfare included in section 302(h) of the CAA identifies both visibility and the broader category of effects on personal comfort and well

- being as effects on welfare. In setting a secondary standard to address visibility impairment, the EPA considers the effect on the public from impairment of visibility as a se parate and distinct welfare effect in its own right. The EPA is not required to translate this into terms of personal comfort and well

- being, as visibility impairment is designated explicitly by Congress as an effect on welfare. While there may be a large degree of overlap among these different welfare effects, the EPA properly focuses on evaluating all of the information before the Agency on the effect visibility impairment has on the public, whether or not this impairment would also be categorized

as havi ng an adverse effect on personal comfort and well

- being. It is in the context of all of this information that the EPA makes the judgment as to the appropriate degree of protection from known and anticipated adverse effects on the public from visibility imp airment. The EPA recognizes that there is uncertainty about the degree of adversity to the public welfare associated with impairment of VAQ. However a secondary standard is designed to provide protection from "known or anticipated" adverse effects, and a bright line determination of adversity is not required in judging the requisite degree of protection under section 109(b)(2 ). Furthermore, the EPA disagrees that it hasInteragency Review Draft

– Do Note Quote or Cite December 12, 2012

PM NAAQS Response to Comments

— Section IV.5 (DRAFT)

12 abandoned its consideration of visibility

- related impacts on the welfare effect of per sonal

comfort and well

- being, as is made clear in the following quote:

Research has demonstrated that people are emotionally affected by low visual air quality, that perception of pollution is correlated with stress, annoyance, and symptoms of depression, and that visual air quality is deeply intertwined with a ‘‘sense of place,’’ affecting people’s sense of the desirability of a neighborhood (U.S. EPA, 2009a, section 9.2.4). Though it is not known to what extent these emotional effects are linked to differ ent periods of exposure to poor visual air quality, providing additional protection against short

- term exposures to levels of visual air quality considered unacceptable by subjects in the context of the preference studies would be expected to provide some degree of protection against the risk of loss in the public’s ‘‘sense of well

- being.’’ (77 FR 38973/1, emphasis added)

The approach taken to address such qualitative, but policy

- relevant, information in this review is the same as in other NAAQS reviews. T he review is initiated with a comprehensive assessment of all possible public health and welfare effects associated with PM in the Integrated Science Assessment. Then policy

- relevant effects for which there is sufficient quantitative information to allow a determination of the change in risks associated with incremental changes in air quality are assessed (in this review, in the Visibility Assessment) and used to provide a quantitative basis to inform the selection of an appropriate range of levels for furt her consideration in the Policy Assessment. In the Policy Assessment, the EPA considers all important policy

- relevant evidence and information, both quantitative and qualitative, in making recommendations regarding the range of policy options appropriate f or the Administrator to consider. It is in the context of all of this information that the Administrator makes her final judgment as to the appropriate degree of protection from known and anticipated adverse effects on the public from visibility impairment

..

(2) Comment: Another issue raised in the comments regarding adversity is the EPA’s decision to use the 50 percent acceptability criterion from the public preference studies in determining candidate protection levels of visibility impairment for the selectio n of a national level of visibility protection. For example, AMC et al. recommended “a 75% acceptability criterion as a target that is in line with protecting the broader public from the negative effects of visibility impairment” (AMC, et al., p. 9)

.

Respo nse: In the Visibility Assessment, the EPA noted that the use of the 50 percent acceptance level for urban visibility was first presented in Ely et al. (1991) (U.S. EPA, 2010b, p. 2

- 5). Ely discussed the use of the 50 percent acceptability criterion as a

r easonable basis for setting a standard to protect visibility in urban areas

.

The standard was determined based on a 50% acceptability criterion, that is, the standard was set at the level of extinction that would divide the slides into two groups: those ju dged acceptable and those judged unacceptable by a majority of the people in the study. The criterion is politically reasonable because it defines the point where a majority of the study participants begin to judge slides as representing unacceptable visib ility. It is alsoInteragency Review Draft

– Do Note Quote or Cite December 12, 2012

PM NAAQS Response to Comments

— Section IV.5 (DRAFT)

13 consistent with psychological scaling theory 4 which indicates that a “true score” exceeds a standard when more than 50% of the “observed scores” exceed that standard. (Ely et al., 1991, p. 11)

As Ely described, the 50 percent acceptability criterion and the preference study conducted by Ely were used as the basis for setting the level of the Denver Visibility Standard in 1990. The same criterion was used in the Phoenix preference study (BBC

rese arch, 2003) and was also used as the basis for setting the level of the Phoenix Visibility Standard in 2003. Thus, after considering the information that served as the original basis for its selection as described in Ely et al., 1991, and given its accepta nce and use in existing visibility programs and the absence of information to support a different acceptability criterion, the EPA continues to conclude that it is reasonable to use the 50 percent acceptability criterion in determining target levels of pro tection from visibility impairment.

d. Comments on Using Preference Studies as Basis for Nationally Uniform Standard

A number of commenters raised concerns regarding the bases for and implications of the differences observed in the preference study result s, concluding that these results were due to regionally varying factors and thus could not be used to set a national standard.

(1) Comment: Some commenters asserted that because the confidence intervals around the four 50 percent acceptability levels do not ov erlap at all, and because there are variations in preference study designs and inherent differences in the visual setting among cities and panels, the four preference curves and their associated 50 percent dv values are city

specific and statistically diff erent. The commenters concluded, therefore, that it was inappropriate to aggregate the 50 th percentile dv values from multiple studies and that they should instead be evaluated individually. Similarly, other commenters expressed the related view that the p reference study results cannot be used to set a national standard for visibility impairment because the results show that visibility preferences vary regionally and/or locally for a variety of reasons . For example :

The ‘one

- size

- fits

- all’ approach...is not viable because it does not account for regional and city

- specific factors that have been made evident in the disparity of preference study data.... It is well known, for example, that the level of light extinction to which people in different areas of the c ountry are accustomed, as well as the urban setting, are the primary factors that affect a person’s visual perception of an urban vista. Thus, the degree to which extinction threshold can be related to human welfare is inevitably regionally

- dependent. (API , Attachment 2, p. 4)

[T]he proposed method falls short because it is not temporally or geographically representative enough to have any meaning.... The uncertainty evidenced in these

studies and the non

- uniformity between the western and eastern vistas mak es it impossible at this time to set an acceptable light extinction value that would

4 J.C. Nunnally, Psychometric Theory , 2nd ed., MacGraw

- Hill, New York, 1978.Interagency Review Draft

– Do Note Quote or Cite December 12, 2012

PM NAAQS Response to Comments

— Section IV.5 (DRAFT)

14 appropriately address visibility concerns in non

- Class I areas. (New York DOH/DEC, pp. 5

- 6)

These commenters suggest that the basis for the differences in preference study results include regionally varying preferences for acceptable visual air quality and/or regionally varying factors, such as relative humidity and other differences

between Eastern and Western areas. These commenters asserted that it would be more appropri ate to develop distinct visibility standards at the state or local level, since it is not possible to select a single level for a national standard to protect visibility across the United States that provides the appropriate degree of protection in locatio ns with distinct characteristics, including areas which lack “important visibility vistas” that might not need standards at all, since flat areas without significant terrain have a limited maximum visual range. (NEDA/CAP, p. 3)

Response: The EPA agrees that the preference curves and the 50 percent dv levels are separate and distinct data points representing four different VAQ preference curves for four unique urban scenes. However, the EPA does not agree that the study results necessarily support a conclusion that preferences are regionally dependent, nor does the EPA consider the fact that the four curves are distinct as a weakness of the approach or a reason that the results cannot be compared. In particular, the EPA notes that the resul ts of Smith and Howell (2009) which show that participants in Houston and Washington, DC did not have significantly different views on acceptable air quality in Washington, DC, provide limited support for the conclusion that people’s preferences differ les s because of where they live and more because of the scene they are viewing. Indeed, these preference study results seem to substantiate what is already known, that people’s preferences for VAQ depend in large part on the characteristics and sensitivity of the scene being viewed. It is to be expected that people looking at four different scenes would have four different preference curves, with four different 50% acceptability values, depending on how changes in VAQ impact the various scenes. Therefore, it m ight also be the case that if the same scene was shown to participants in different regions of the country, a single preference curve for that scene could be drawn. The preference study results showing that people find less air quality degredation acceptab le for sensitive scenes than for less sensitive scenes could be interpreted as validating the appropriateness of the approach for eliciting public preferences. It is the EPA’s view that the differences in the preference study results appropriately reflect the different sensitivities to visibility impairment among the four urban scenes and thus provide useful information about how people’s preferences for visual air quality would change when viewing urban scenes of differing

sensitivities.

In response to th e comments that there are factors that vary regionally that are important to take into account when setting a national standard for visibility protection, the EPA agrees, and notes that it has clearly taken these into account. Section VI.A above regarding the history of the secondary PM NAAQS review discusses the evolution of the EPA’s understanding regarding the regional differences in PM concentrations, relative humidity and other factors. As a result, the current review has gone to great lengths toInteragency Review Draft

– Do Note Quote or Cite December 12, 2012

PM NAAQS Response to Comments

— Section IV.5 (DRAFT)

15 addre ss these factors, leading to the EPA’s proposal to use the IMPROVE algorithm to calculate light extinction in order to take into account the varying effects of relative humidity and speciated PM. While this approach does not result in a uniform level of

am bient PM 2.5 , it does ensure a nationally uniform level of visibility protection. The EPA refers the reader to other sections of the final rule, including sections VI.B.1.a, VI.B.1.c, VI.C.1.b and VI.C.1.f, and other sections of the Response to Comments doc ument for a more detailed response as to how it is taking these variables into account.

(2) Comment: The Arizona DEQ stated that it was inappropriate to use the Phoenix study for purposes of establishing a level for a national standard:

The proposed seconda ry visibility standard does not integrate our measurement method or mirror our index scale, essentially making the program incomparable with the proposed standard. Four urban visibility studies were cited in the proposed rule as a consideration in developi ng the level of the secondary PM 2.5 standard. The recommended level for the proposed standard, 28 or 30 deciviews, and corresponding calculation method excludes extinction by coarse particulate matter and Rayleigh scattering. The cited studies were conduct ed considering total light extinction; including extinction resulting from particulate matter and Rayleigh scattering. Visibility impairment due to coarse particulate matter can be an important contributor in Arizona, specifically in the Phoenix area where ongoing measurements have been made. Therefore, ADEQ believes that the proposed levels of the secondary visibility standard are inconsistent with applicable urban studies. (Arizona DEQ, p. 2)

Response: The EPA understands that each of the four preference study areas has a different mix of pollutants and that across the studies a variety of methods for measuring and linking ambient PM concentrations to images (either photos or WinHaze) of differing haze levels at a given urban scene were employed. However, as discussed above in (2) in subsection i., the EPA conducted a detailed assessment (77 FR 38893, 77 FR 38974

- 75)

of the differences and similarities that existed between the four studies and concluded that it was still appropriate to compare them and cons ider their results together. The CASAC also reached the same conclusion (Samet, 2010b, iii). In regard to the commenter’s concern about contributions from coarse particles and Rayleigh, the EPA further notes that participants in the Phoenix and other studi es were not asked to break out their preferences based on how much of the haze was caused by contributions from coarse particles, Rayleigh scattering or fine particles. Instead, as noted above in comment (2) in subsection ii., Phoenix study participants we re simply asked to identify levels of visual air quality that they would consider “ unreasonable or objectionable”, no matter what the source. Based on this information, Phoenix was able to come up with a target level of VAQ, even though pollutant mixes var y temporally and spatially, even within the Phoenix urban area. Since the goal of the secondary PM 2.5 NAAQS is to provide a uniform national level of public welfare protection from PM 2.5 related visibility impairment, the form of the EPA’s proposed seconda ry standard was selected to allow each area to achieve that level of protection in a way that addresses its own unique mix of contributing pollutants. Therefore, the EPA disagrees with the commenter that it isInteragency Review Draft

– Do Note Quote or Cite December 12, 2012

PM NAAQS Response to Comments

— Section IV.5 (DRAFT)

16 inappropriate to use the results of the Phoeni x study to inform a range of candidate protection levels for consideration by the Administrator in selecting a national level of visibility protection. Instead, the EPA believe the Phoenix results form an important data point that shed light on pub l ic pref erences for visibility in western areas with urban

scenes that include distant mountains.

(3) Comment: Similar to comment (2) above, Colorado DPHE expressed concern with the EPA’s use of the Denver study results in a national context. In particular the comm enter

states:

The proposed secondary visibility standard relies in part on the Denver urban visibility standard as supporting justification for establishing a new secondary standard. However, the Denver visibility standard was originally developed in

1989 in response to a bipartisan directive from the legislature to meet a statutory deadline of January 1990 for a standard based solely on aesthetics. The Denver visibility standard, set at about 20 deciviews (32 miles or 0.076/km), was based on a survey of 2 14 Denver citizens that evaluated 25 slides of the ‘Brown Cloud’, a publically understood name for the strong inversions that occur in the Denver metropolitan area. These slides were specifically referenced to data obtained using a transmissometer. Thus, t his standard is based on a specific view of Denver and specific transmissometer location and path and is applicable only to this location. This constitutes a limitation with its applicability to a national secondary standard, particularly a proposed standa rd that does not use a direct measurement method. ... Visibility from year to year is highly variable.... [A]dditional studies are necessary to establish a secondary NAAQS standard, or what might ultimately become multiple standards, across the United States ” (Colorado DPHE, p. 2

- 3).

Response: The Colorado comment raises some of the same concerns as the Arizona comment and the EPA refers the reader to its response to comment (2) above. In addition, this commenter states that the visibility standard is “based solely on aesthetics”. This is not a significant difference with the proposed secondary visibility PM NAAQS. While qualitative information was also available regarding personal comfort and well being, the proposed range of levels considered by the Administ rator was derived from the quantitative information regarding public preferences for the aesthetic quality of the urban view. This commenter also points out that the slides presented to the study participants included images of the Brown Cloud that occurs under conditions of a strong inversion. The EPA notes that while this condition does not happen everywhere, there are other urban areas that experience it and therefore preference information that takes such a phenomenon into account provides useful inform ation for the Administrator to consider. The commenter further notes that “this standard is based on a specific view of Denver and specific transmissometer location and path and is applicable only to this location.” In spite of this, however, the EPA obser ves that Denver still went ahead and established a standard that applies to the entire metropolitan area, even though there are many other views and paths that could be evaluated. Likewise, the EPA believes it is possible to

select a national level of visi bility protection based on a variety of disparate data points for urban visibility preferences, because when taken together, they provide usefulInteragency Review Draft

– Do Note Quote or Cite December 12, 2012

PM NAAQS Response to Comments

— Section IV.5 (DRAFT)

17 information regarding the range of scene and site characteristics that can influence public perception of accep table VAQ. While the EPA agrees that additional studies would be helpful and provide more useful data points to consider, it does not agree that they are necessary to establish a secondary standard.