# Endangerment and Cause or Contribute Findings for Greenhouse Gases Under Section 202(a) of the Clean Air Act:

EPA’s Response to Public Comments

Volume 1: General Approach to the Science and Other Technical Issues

# General Approach to the Science and Other Technical Issues

U.S. Environmental Protection Agency Office of Atmospheric Programs Climate Change Division Washington, D.C.

### FOREWORD

This document provides responses to public comment s on the U.S. Environmental Protection Agency’s (EPA’s) Proposed Endangerment and Cause or Cont ribute Findings for Greenhouse Gases Under Section 202(a) of the Clean Air Act, published at 74 FR 18886 (April 24, 2009). EPA received comments on these Proposed Findings via mail, e-mail, and facsimile , and at two public hearings held in Arlington, Virginia, and Seattle, Washington, in May 2009. Copies of all comment letters submitted and transcripts of the public hearings are available at the EPA Do cket Center Public Reading Room, or electronically through http://www.regulations.gov by searching Docket ID EPA-HQ-OAR-2009-0171.

This document accompanies the Administrator’s final Endangerment and Cause or Contribute Findings for Greenhouse Gases Under Section 202(a) of the Clean Air Act (Findings) and the Technical Support Document (TSD), which contains the underl ying science and greenhouse gas emissions data.

EPA prepared this document in multiple volumes, with each volume focusing on a different broad category of comments on the Proposed Findings. Th is volume of the document provides responses to public comments regarding the general approach to the science and other technical issues.

In light of the very large number of comments received and the significant overlap between many comments, this document does not respond to each co mment individually. Rather, EPA summarized and provided a single response to each significant argumen t, assertion, and question contained within the totality of comments. Within each comment summary, EPA provides in parentheses one or more lists of Docket ID numbers for commenters who raised particul ar issues; however, these lists are not meant to be exhaustive and EPA does not individua lly identify each and every commenter who made a certain point in all instances, particularly in cases where multiple commenters expressed essentia lly identical arguments.

Several commenters provided additional scientific litera ture to support their arguments. EPA’s general approach for taking such literature into consideration is described in Volume 1, Section 1.1, of this Response to Comments document. As with the comments, there was overlap in the literature received. EPA identified the relevant literature related to the significant comments, and responded to the significant issues raised in the literature. EPA does not indivi dually identify each and every piece of literature (submitted or incorporated by reference) that made a certain point in all instances.

Throughout this document, we provide a list of re ferences at the end of each volume for additional literature cited by EPA in our responses; however, we do not repeat the full citations of literature cited in the TSD.

EPA’s responses to comments are generally provided immediately following each comment summary. In some cases, EPA has discussed responses to specifi c comments or groups of similar comments in the Findings. In such cases, EPA references the Findings rather than repeating those responses in this document.

Comments were assigned to specific volumes of th is Response to Comments document based on an assessment of the principal subject of the commen t; however, some comments inevitably overlap multiple subject areas. For this reason, EPA encourages the public to read the other volumes of this document relevant to their interests.



### iii

## TABLE OF CONTENTS

1.0 General Approach to the Science and Other Technical Issues 1.1 Use of IPCC, CCSP, USGCRP, and NRC Reports as the Primary Scientific Basis...........................................................................................................1

1.1.1 Specific Comments on EPA’s Use of IPCC Reports 1.1.2 Specific Comments on EPA’s Use of CCSP and USGCRP Reports 1.2 Level of Scientific Certainty ....................................................................................32 1.3 Treatment of Adaptation and Emissions Mitigation .............................................46 1.4 Consideration of Net (Adverse and Beneficial) Effects........................................51 1.5 Information Quality Act Discussion .......................................................................52

Appendix A: IPCC Principles and Procedures.......................................................................66

Appendix B: USGCRP/CCSP Procedures and Responsibilities...........................................88

Appendix C: NRC Report Development Procedures ...........................................................118

Acronyms and Abbreviations

ANPR Advance Notice of Proposed Rulemaking AR4 Fourth Assessment Report CAA Clean Air Act CCSP U.S. Climate Change Science Program CO 2 carbon dioxide CRU Climate Research Unit EPA U.S. Environmental Protection Agency IPCC Intergovernmental Panel on Climate Change IQA Information Quality Act GHG greenhouse gases NIPCC Nongovernmental International Panel on Climate Change NOAA National Oceanic and Atmospheric Administration NRC National Research Council RCF Request for Correction SAP synthesis and assessment product SDWA Safe Drinking Water Act SPM Summary for Policymakers TSD Technical Support Document USGCRP U.S. Global Change Research Program



### iv1.0 General Approach to the Science and Other Technical Issues

1.1. Use of IPCC, CCSP, USGCRP, and NRC Reports as the Primary Scientific Basis

Comment (1-1) : A large number of commenters (0309.1, 0322, 0487, 0499, 0509, 0538, 0664, 0708.1, 0798, 0800.1, 0921, 1005.1, 1309.1, 1616.1, 1924, 1927, 1961, 2697.1, 2759, 2813.1, 2895, 2898.1, 2916.1, 2972.1, 3013, 3053, 3136.1, 3145.1, 3187.5, 3219.1, 3224, 3250, 3252.1, 3286.1, 3291.1, 3303, 3307.1, 3316.1, 3323.1, 3330.1, 3372.1, 3373, 3379.1, 3389, 3394.1, 3397, 3407, 3423.2, 3427.1, 3433.1, 3440.1, 3446.1, 3446.3, 3449.1, 3452.1, 3473.1, 3497.1, 3533.1, 3548.1, 3560.1, 3567.1, 3596.1, 3596.2, 3603.1, 3630.1, 3681.1, 3702.1, 3707.1, 3722, 3729.1, 3747.1, 3748.1, 3764.1, 3915, 4003, 4032, 4041.1, 4172, 4288, 4446, 4509, 4932.1, 5158, 9061.1, 9680, 9863, 9970, 10158, 10160, 10524, 10956, 11029, 11288, 11315, 11335, A18, 11410, 11466.1, and 11493) generally argued that EPA inappropriately relied almost exclusively on information from the assessment literature in developing the Technical Support Document (TSD). The commenters said that in taking this appr oach EPA failed to use the best available science and consider other important studies related to the topics presented in the TSD.

Response (1-1) : See Section III.A. of the findings, “The Science on Wh ich the Decisions Are Based,” for our response to comments on the use of the assessment literature. Please see responses to co mments in this section regarding specific issues raised by comme nters related to this general comment.

EPA recognizes that scientific research is very activ e in many areas addressed in this document. Many commenters sent scientific articles, reports, and synthesis studies to the EPA as part of their technical and scientific comments. These submissions were categori zed as either ‘submitted reference material’ or ‘materials incorporated by reference.’ Submitted refe rence materials are those materials that commenters provided with their comments. Materials incorporated by reference are those materials that were referred to in comments (e.g., article and report titles, book titl es, or website links) but that were not submitted to EPA. EPA made every effort to retrieve and review publicly available (i.e., available free of charge via the internet or standard library access) materials incorporated by reference, particularly those in the peer reviewed literature yet not in the peer reviewed assessment literature.

As noted in the Foreword to each volume of th e Response to Comments document, EPA did not individually respond to each submitted comment. Inst ead, we grouped similar comments together and addressed them:

In light of the very large number of comments received and the significant overlap between many comments, this document does not respond to each comment individually. Rather, EPA summarized and provided a single response to each significant argument, assertion, and question contained within the totality of comments. Within each comment summary, EPA provides in parentheses one or more lists of Docket ID numbers for commenters who raised particular issues; however, these lists are not meant to be exhaus tive and EPA does not i ndividually identify each and every commenter who made a certain point in all instances, particularly in cases where multiple commenters expressed e ssentially identical arguments.

Similarly, we did not individually respond to each an d every piece of literature submitted or incorporated by reference in comments. As explained in the Foreword:

As with the comments, there was overlap in the literature received. EPA identified the relevant literature related to the significant comments, a nd responded to the significant issues raised in the



### 1literature. EPA does not individua lly identify each and every pi ece of literature (submitted or incorporated by reference) that ma de a certain point in all instances.

We categorized the literature we received for purpo ses of its consideration de pending on whether or not it was peer-reviewed, and, if so whether it was refe renced in the assessment literature summarized in the TSD. We considered such literature as having alr eady been incorporated into the peer-reviewed assessment literature. In many cases, we note this in our response to comme nts, and in some cases we also discuss such literature in our responses if it provides important additional detail or information. If peerreviewed literature was not referenced in the assess ment reports, we addressed the major findings in the relevant comment response. As noted above, this doe s not imply that every specific piece of literature related to a particular issue was mentioned in that co mment response. We also reviewed the literature that was not peer-reviewed when responding to signi ficant comments. In many cases we discussed the findings or assertions of such literature in our comment responses, but noted that the literature was not peer-reviewed, and it was accorded less weight as a result.

The Agency recognizes the potential importance of ne w scientific research, and we note within this Response to Comments document where new scientific information (from either new assessment reports or individual studies) reinforce, refine, or correct specific issues summarized within the TSD. Together, the updated final TSD and EPA’s responses to public comments (contained in the Findings and the 11 volumes of the Response to Comments document) c onstitute the scientific summary on which the Administrator has made her determination. We note that many valuable insights were obtained through the review of comments as reflected in the Findings , the TSD, and the Response to Comments document. However, we did not identify specific issues or ne w literature that alters, in a significant way, our understanding of how greenhouse gas emissions affe ct the climate and how climate change may reasonably be anticipated to endanger public health or welfare.

Comment (1-2) : A number of commenters (3324.1, 3394.1, 3429.1, 3440.1, 3481.1, 3533.1, 3579.1, 3596.1, 3707.1, 3747.1, 3764.1, 3915, 4509, 5716, 8624, 10197, 10948, and 11335) argue that the Administrator did not reference any documents in her decision-making pro cess that offered alternative views on the scientific basis for causes of climate change. Commenters recommend that EPA provide a more complete discussion of global warming science by using information outside the assessment literature, including studies that do not support its conclusions and st udies submitted to the docket through the public comment period for the Proposed Findings.

Response (1-2) : We disagree that the Administrator did not consid er alternative views on climate change science and impacts. Such a contention incorrect ly implies that all individual studi es reviewed and incorporated into the assessments of U.S. Global Change Research Program (USGCRP), U.S. Climate Change Science Program (CCSP), Intergovernmental Panel on Clim ate Change (IPCC), and National Research Council (NRC) support uniform conclusions on the different elements of greenhouse gas (GHG) and climate change science, and that the many thousands of studi es reviewed and reflected in these major assessment works represent a narrow view of GHG and climate cha nge science. In fact, these assessment reports look at the range of the scientific literature without “cherry-picking” and it is EPA’s conclusion that by placing primary reliance on the major assessment reports, we ha ve ensured that the determinations are based on reports that have considered and weighed all views. EPA relied on the major peer-reviewed assessment reports in developing the TSD precisely to avoid an over-reliance on and narrow consideration of individual studies and to ensure that the Administ rator’s decision would be based on a comprehensive assessment of the scientific literature. EPA has determin ed that the approach taken provided the high level



### 2of transparency and consistency outlined by EPA’s Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by the Environmental Protection Agency.

Regarding the recommendation that EPA provide a mo re comprehensive discussion and consideration of global warming science and literature that is representative of “alternative views,” we have done this through our review of comments and literature subm itted, the associated revision of the TSD, and the development of the eleven volumes of the Response to Comments document. See Section III.A. of the Findings, “The Science on Which the Decisions Are Based,” for our response to comments on the use of the assessment literature and previous responses in this section regarding our treatment of new and additional scientific literature provided through th e public comment process. For EPA’s responses to comments and literature provided on specific climat e science issues in the TSD, please refer to the appropriate Response to Comment volumes.

Finally, we note that IPCC, USGCRP/CCSP, and NRC make considerable effort to ensure that their assessment reports reflect a balance of perspectives re garding the state of the science. We note that the CCSP report Guidelines for Producing CCSP Synthesis and Assessment Products state that the reports should “identify disparate views that have significan t scientific or technical support” (CCSP, 2004). We refer commenters to Appendix B of this volume for the full listing of these Guidelines . Further, NRC (National Academies, 2006) report development procedures state that:

[C]areful steps are taken to convene [report development] committees that meet the following criteria:

An appropriate range of expertise for the task. The committee must include experts with the specific expertise and experience needed to address the study’s statement of task. One of the strengths of the National Academies is the tradition of bringing together recognized experts from diverse disciplines and backgrounds who might not otherwise collaborate. These diverse groups are encouraged to conceive new ways of thinking about a problem.

A balance of perspectives . Having the right expertise is not sufficient for success. It is also essential to evaluate the overall compos ition of the committee in terms of different experiences and perspectives. The goal is to ensure that the relevant points of view are, in the National Academies’ judgment, reasonably balanced so that the committee can carry out its charge objectively and credibly.

Screened for conflicts of interest. All provisional committee members are screened in writing and in a confidential group discussion about possible conflicts of interest. For this purpose, a “conflict of interest” means any financ ial or other interest which conflicts with the service of the individual because it could significantly impair the individual’s objectivity or could create an unfair competitive advantage for any person or organization. The term “conflict of interest” means something more than individual bias. There must be an interest, ordinarily financial, that could be directly affected by the work of the committee. Except for those rare situations in which the National Academies determine that a conflict of interest is unavoidable and promptly and publicly disclose the conflict of interest, no individual can be appoi nted to serve (or continue to serve) on a committee of the institution used in the development of reports if the individual has a conflict of interest that is relevant to the functions to be performed.”



### 3We refer commenters to Appendix C of this volume for more information on NRC report development process. In addition, please see response to comment s below regarding the IPCC report development process and procedures for ensuring a balance of perspectives.

Comment (1-3) : Several commenters (2895, 2916.1, 3145.1, 3330.1, 3497.1, 3533.1, 3630.1, 3747.1, and 10524) argue that EPA cherry-picked specific studies from the sc ientific literature in making its conclusions.

Response (1-3) : EPA disagrees with the comment that the Agency selectively chose studies to serve as the basis for the endangerment finding, as addressed in our response to comment 1-2.

Comment (1-4) : A large number of commenters (0245, 0636, 0655, 2898.1, 2972.1, 3145.1, 3187.1, 3214.2, 3224, 3316.1, 3323.1, 3347.4, 3360, 3373, 3394.1, 3397, 3423.2, 3427.1, 3433.1, 3440.1, 3452.1, 3482.1, 3533.1, 3597, 3630.1, 3681.1, 3722, 3729.1, 3747.1, 4003, 4041.1, 4168, 4172, 4932.1, 5158, 7020, 9060.1, 9680, 9863, 10158, 10577, 10956, 11046, and 1 1165) argue that EPA should have conducted its own assessment of climate data and research in developing the Proposed Findings, instead of relying on the body of assessment literature. The commenters argue that the assessment reports do not represent the best available science for a number of reasons, including inappropriate review process, failure to meet information quality guidelines, etc. Commenter 3567.1 states that for the Proposed Findings, EPA should have commissioned a body of independent scientists, engineers, and statisticians to evaluate the data, methods, and conclusions of the most important research.

Response (1-4) : See Section III.A. of the Findings, “The Science on Wh ich the Decisions Are Based,” for our rationale on the approach to the scientific literature and our d ecision that it was not necessary nor logical for EPA to conduct an additional and separate review of the underl ying climate data and research. We respond to the specific comments regarding the assessment literature (e.g., inappropriate review process) and the information quality guidelines separately below.

Comment (1-5) : A commenter (11458) argues that technically and traditi onally, EPA has no purview or expertise in any of the various areas of concern cited in the Proposed Fi ndings (i.e. storms, droughts, sea levels, crop yields, etc.). The commenter states that climate change is not an issue where there is some dose-dependent biological response to a traditional pollutant; it is a “complex physical response (complete with positive and negative feedback) by an entire planet!” The commenter states that because EPA “necessarily has limited expertise, knowledge base, or experience” in these areas, it is inappropriate for EPA to move forward with the Findings.

Response (1-5) : We disagree with the commenter’s assertion that EPA does not have purview or expertise in the climate science issues discussed in the TSD. As stated in the Findings, EPA has been a very active part of the United States Government climate change research enterprise, and has taken an active role in the reviewing, writing, and approval of the USGCRP/CCS P assessment reports. EPA was the lead agency for three significant reports under the USGCRP and recently completed an assessment addressing the climate change impacts on air quality in the United States. EPA was also involved in a review of the IPCC Fourth



### 4Assessment Report (AR4), and in particular took part in the approval of the summary for policymakers for the Working Group II volume, Impacts, Adaptation, and Vulnerability . We note that the specific climate science issues raised by the commenter (e.g., storms, droughts, sea levels, crop yields) and the positive/negative feedbacks are all covered in great detail within these assessment reports.

We did not develop new science to support the finding, but rather relied primarily on the conclusions of the major assessment reports of USGCRP/CCSP, IP CC, and NRC and the evaluation of the public comments received. See Section III.A. of the Findings , “The Science on Which the Decisions Are Based,” for our response to comments on the rationale for our use of the assessment lite rature. The use of the assessment literature capitalizes on the substantial expertise and experience that went into the development of those reports. No other source of information provides such a comprehensive and indepth analysis across such a large body of scientific studies.

Comment (1-6) : Several commenters (3252.1, 3286.1, 3379.1, 3394.1, 3449.1, 3705.1 and 3579.1) argue that the TSD relies upon sources of information that do not adequa tely address the scientific issues relevant under the Clean Air Act (CAA), because the IPCC assessment reports do not address United States–specific effects in isolation, were published several years ago, a nd do not follow information quality guidelines.

Response (1-6) : See EPA’s responses in this volume which address the comments regarding the geographic scope of IPCC’s findings, the timeliness and relevancy of these reports, and their consis tency with information quality guidelines.

The endangerment analysis for greenhouse gases under th e CAA requires that EPA examine the extent to which the GHGs constitute the air pollution that may be reasonably anticipated to endanger public health or welfare. Please refer to Section IV.A. of the Findings (“The Air Pollutio n Consists of Six Key Greenhouse Gases”) for the Administrator’s rationale as to why the six well-mixed GHGs are defined as the air pollution for purposes of the endangerment finding under Section 202(a) of the CAA. The Findings discuss in detail the information that is rele vant to the determination and how the Administrator has interpreted it in deciding whethe r the air pollution is reasonably antic ipated to endanger public health or welfare. The scientific literature as synthesized in the TSD provides exactly the kind of information that can help inform these issues. For example, the TSD summarizes the conclusions of the assessment reports with respect to: 1) current emissions of GHG emissions; 2) how these emissions are changing the composition of the atmosphere; 3) how such changes in the atmosphere are affecting the global and regional climate; and 4) the potential impacts of such changes in climate on human health and welfare, for current and future generations. In its scope and quality, the assessme nt literature is relevant and appropriate for addressing the scientific issues under th e CAA. Furthermore, the most recent study from the assessment literature and synthesize d in the TSD was released in June, 2009. That study, Karl et al. (2009), provides both U.S.-focused information as well as re-affirms many of the key findings from the assessment literature from previous years (e.g., 2007).

Comment (1-7) : A commenter (4529) argues that the assessment reports on climate change used by EPA do not follow the scientific method and therefore provide an inappr opriate basis for making an endangerment finding.

Response (1-7) :



### 5For EPA’s responses to comments and literature provide d on specific climate science issues in the TSD, please refer to the appropriate Response to Comment volumes.

We disagree with the commenter’s general claim th at the assessment reports on climate change do not follow the scientific method. EPA notes that the comment did not provide specific examples or other support for this assertion, nor did the commenter pr ovide detail as to how the assessment literature does not follow the scientific method. We are aware of no evidence to support the commenter’s charge. To the contrary, EPA has found that the assessment reports a ppropriately applied the scientific method by using multiple lines of climate science ev idence to formulate and test hypotheses, investigate cause and effect relationships in observations, and evaluate new knowledge and advancements.

The assessment reports summarized in the TSD were prepared following rigorous and transparent processes addressing such issues as the nomination and selection of authors, the caliber of literature reflected in the assessment, and the processes for re view and revision of reports. The TSD provides a general description of these procedures in Section 1(b), and Box 1.1, and our responses to specific comments on these issues are provided in Sections 1.1.1 and 1.1.2 below.

Comment (1-8) : At least one commenter (3307.1) asserts that EPA’s “fail[ure] to consider reliable” scientific data made available after the publication of the Advance Noti ce of Proposed Rulemaking (ANPR) “flouts” the 1997 Report of the Presidential Commission on Risk Assessment and Risk Management . They also assert that EPA fails “to appropriately explain and incorporat e relevant scientific data” as suggested by the 2009 Presidential Memorandum on Scientific Integrity.

Response (1-8) : EPA has considered all relevant inform ation in the public record for this rulemaking. In its consideration, EPA has fully complied with all legal requirements, and has acted consistently with the 1997 report and 2009 Memorandum commenter cites. We note that the 1997 Report states that assessing risks should allow for “ incorporation of important new information th at may emerge at any stage of the risk management process.” The 2009 Memorandum describes that “there should be transparency in the preparation, identification, and use of scientific a nd technological information in policymaking” and that the scientific information used in policy decisions “should be subject to well-established scientific processes, including peer review where appropriate.” As discussed in the Findings and the Response to Comments document, EPA’s approach to the science di d allow for the incorporation of important new information (e.g., more recent assessment reports and public comment), and followed well-established scientific processes (e.g., the assess ment reports were peer-reviewed). For a fuller discussion of the science considered, as well as the reliability of the science used, please see Section III.A. of the Findings, “The Science on Which the Decisions Are Based,” and responses in this section of Volume 1.

Comment (1-9) : A commenter (0719) discusses how it is important to recognize that broad assessments, such as IPCC and CCSP reports, are too conservative in projecting climate change impact s and typically represent a kind of lowest common denominator in scientific judgment. The commenter states that EPA should not always wait for science to provide overwhelming certainty and that judgment must be exercised to avoid catastrophic risks even when the science is still incomplete.

Response (1-9) :



### 6EPA notes that through our evaluation of the scientific literature and review of public comments, we have determined that the TSD provides a sound foundation for the Administrator’s endangerment analysis. We do not find the assessment reports to be too conservative. Rather, we find that they represent the best available science because they review and analyze the most recent science and evaluate uncertainties in making their conclusions. We received many comment s on the legal requirements for the endangerment determination, and our response on the issue of wh ether the Administrator need wait until harm has occurred and the precautionary and preventive nature of the statutory language is provided in Section II. A. of the Findings, “Legal Framework for this Action.”

Comment (1-10) : Several commenters (1309.1, 2750, 3187.5, 3316.1, 3394.1, 3440.1, 3548.1, 3729.1, 3747.1, 7020, and 11459) argue that EPA’s external peer-review pr ocess was inadequate because the federal expert reviewers were involved with developing the IP CC and CCSP reports upon which the endangerment finding is based and therefore not objective.

Response (1-10) : EPA disagrees that review of the scientific and technical information contained in the TSD was inadequate. EPA did not develop new science as part of this action and instead summarized the existing peer-reviewed assessment literature. The decision to have 12 federal experts review the TSD was reasonable and appropriate and we disagree that the reviewers were not objective as a result of their involvement with the IPCC and CCSP. Given our appr oach to the scientific literature (described in Section III.A of the Findings), the purpose of the fe deral expert review was to ensure that the TSD accurately summarized the conclusions and associated uncertainties from the assessment reports. The federal experts were ideal candidat es because they have contributed si gnificantly to the body of climate change literature and played active roles in IPCC and CCSP—therefore making them experts on various aspects of climate science and very familiar with th e underlying literature and state of the science. Furthermore, the federal climate change experts represent a range of technical specialties that span the range of topics covered in the TSD and covered by the range of topics that the Administrator needed to consider. In addition, the federal experts were not involved with de veloping the TSD or Findings in any way other than their review roles. Finally, we note th at the federal expert review was only one part of a much larger process of developing the TSD from 2007 until the present. In addition to the three rounds of technical review by the 12 federal experts, the TSD has also gone through three rounds of internal EPA review, and two rounds of public comment.

Comment (1-11) : Two commenters (3187.5, 3729.1, and 3764.1) argue that EPA should once again review the comments received on the TSD for the (ANPR) on Regulating Greenhouse Gases Under the Clean Air Act regarding the use of IPCC and CCSP reports.

Response (1-11) : EPA received a number of comments specifically focused on the TSD during the 120-day public comment period for the 2008 ANPR. We reviewed these comments and revised and updated the version of the TSD that was released with the Proposed Fi ndings on endangerment and cause or contribute in April 2009. Thus, many of the comments received on the ANPR were reflected in the version of the proposal and the TSD released in April 2009, whic h served as the underlying scientific and technical basis for the proposal. For the purposes of this action, EPA has once again reviewed the comments received on the ANPR regarding the use of the assessment literature in developing the TSD. Most of these comments were also submitted during the notice a nd 60-day comment period on the Proposed Findings,



### 7and EPA’s responses to those resubmitted comments ar e provided in the relevant Response to Comment volumes. EPA used an ANPR to notify the public about actions that were broadly being considered and to gather information on specific topics and the appropria te scope for potential rulemaking. It is not clear whether commenters are suggesting that existing law prescribes such a requirement; however EPA disagrees with that interpretation; there is no legal requirement for the Agency to formally respond to comments filed on the ANPR and not resubmitted during this public comment period.

Comment (1-12) : Several commenters (1924, 2898.1, 3214.1, 3330.1, 3389, 3446.2, 3560.1, 3679.1, 3748.1, 3969.1, and 4172) argue that EPA should base its endangerment finding on the recent Nongovernmental International Panel on Climate Change (NIPCC) report entitled Climate Change Reconsidered, instead of IPCC and CCSP reports.

Response (1-12) : EPA has reviewed and considered the NIPCC report and found that it lacks the rigorous procedures and transparency required to serve as a foundation for th e endangerment analysis. A review of the NIPCC Web site indicates that the NIPCC report was devel oped by “two co-authors” and “35 contributors and reviewers” from “14 countries” ( http://www.nipccreport.org/index.html ). The organization does not appear to have established any procedures for author selection and provides no evidence that a transparent and open public or expert review wa s conducted. Thus, the NIPCC’s approach stands in sharp contrast to the clear, transparent, and open procedures of the IPCC, CCSP, USGCRP, and NRC. Relying on the work of the major assessment reports is a sound and reas onable approach. See Section III.A. of the Findings, “The Science on Which the Decisions Are Based,” for our response to comments on the use of the assessment literature and previous responses in this section regarding our treatment of new and additional scientific literature provided through the public comment process.

Although EPA sees no reason to base the endangerment analysis on the NIPCC, we did thoroughly review the report and the associated references. Fo r EPA’s responses to comments and literature provided on specific climate science issues in the TSD, including the work of the NIPCC, please refer to the appropriate Response to Comment volumes.

Comment (1-13) : Several commenters (3323.1, 4003, 4041.1, 4932.1, 5158, and 11493) attached the following reference document to their comment: Proposed NCEE Comments on the Draft Technical Support Document for Endangerment Analysis for Greenhouse Gas Emissions Under the Clear Air Act . This memorandum was developed by Alan Carlin in EPA's National Ce nter for Environmental Economics. The commenters argue that EPA has accepted the findings reached by outside groups, such as IPCC and CCSP, as being correct without undertaking a careful and critical ex amination of the conclusi ons and documentation in the Carlin document. The document itself and sever al of the commenters suggest that EPA conduct a review of all available climate change science befo re moving forward with th e endangerment finding.

Response (1-13) : As noted in our response to Comment 1-1, EPA h as reviewed new and additional scientific literature received through public comment. This review includ es the Carlin document; see the relevant Response to Comments volumes for our responses to specific scientific comments raised in that document.

See Section III.A. of the findings, “The Science on Wh ich the Decisions Are Based,” for our response to comments on the use of the assessment literature and EPA’s decision not to conduct a new and separate



### 8assessment of the vast body of underlying science. Finally, see Volume 11, Sec tion 11.6.4 (Transparency) for our response to further comments regarding the Carlin document.

1.1.1 Specific Comments on EPA’s Use of IPCC Reports

Comment (1-14) : Commenters (1309.1, 2813.1, 3187.5, 3330.1, 3389, 3429.1, 3440.1, 3597, 3722, 4032, 4288, 9061.1, 10196, 10197, 11288, 11315, 11409, 11046, and 11410) argue that the process and structure that IPCC employs in developing its assessment reports yields flawed reports. Specifically, several commenters (0664, 3187.5, 3250, 3330. 1, 3372.1, and 4172) add that IP CC report development process is not transparent. Several of these commenters argue that not all of the authors or governments that participated in the report development process agree with all of th e conclusions and that consensus was never reached. A few commenters (1309.1, 3187.5, and 11409) argue that the IPCC information quality process and system for engaging public comments is flawed, as the IPCC ignores the best available science and comments that have been submitted during public comment. Several commenters (0582, 1309.1, 3187.5, 3747.1, 4172, and 11 409) argue that the review process IPCC u ses is not objective, transparent to the public, or effective in ensuring that the best available information is used.

Response (1-14) : See Section III.A. of the Findings, “The Science on Wh ich the Decisions Are Based,” for our response to comments on the use of the assessment literature and previous responses in this section regarding our treatment of new and additional scientific literature provided through the public comment process. For EPA’s responses to comments and literature provide d on specific climate science issues in the TSD, please refer to the appropriate Response to Comment volumes.

We refer the commenters to Appendix A of this volume, which contains IPCC’s Principles Governing IPCC Work (2006), IPCC’s Procedures for the Preparation, Revi ew, Acceptance, Adoption, Approval, and Publication of IPCC Reports (1999), and IPCC’s Guidance Notes for Lead Authors of the IPCC Fourth Assessment Report on Addressing Uncertainties (2004). Regarding the role of the IPCC, we quote from their Principles (IPCC, 2006) which states that:

The role of the IPCC is to assess on a compre hensive, objective, open and transparent basis the scientific, technical and socio-econom ic information relevant to understanding the scientific basis of risk of human-induced climate change, its potential impacts and options for adaptation and mitigation. IPCC re ports should be neutral with respect to policy.

EPA has both evaluated and participated in the de velopment and review of IPCC reports, and the IPCC process is transparent and rigorous. The IPCC ensures scientific credibility and legitimacy of its reports by fairly representing the range of scientific opini ons on climate change, and the IPCC provides multiple opportunities for input from experts along the entire spectrum of opinions. Public comment periods for the three Working Group reports that compose IPCC (2007) were held to provide the international community with opportunities to review and comment . In the U.S., the CCSP announced public comment opportunities in the Federal Register and on the CCSP Web site (see list of CCSP announcements and FR notices at http://www.climatescience.gov/Library/ipcc/default.htm ) for each of the IPCC (2007) Working Group reports and the Synthesis Report. CCSP also solicited nominations of author and reviewers for the IPCC (2007) reports using the same communications vehicles. Finally, IPCC provides the public with access to comments received and how they were res ponded to for transparency purposes. Please see the excerpt from IPCC’s Procedures (IPCC, 1999) at the end of this response which describes in detail the



### 9procedures for author selection, report drafting, review by governments and experts, final report preparation, and formal approval and acceptance. Appendix A of this volume contains IPCC’s Procedures (IPCC, 1999) in its entirety.

Regarding the comment that the authors involved in producing IPCC (2007) and the governments charged with its review and acceptance did not agree with a ll of the conclusions, we note that the IPCC has a robust and transparent process for reaching consensus and properly documenting instances where consensus is not met. IPCC’s Principles (IPCC, 2006) state that:

In taking decisions, and approving, adopti ng and accepting reports, the Panel, its Working Groups and any Task Forces shall use all best endeavors to reach consensus. If consensus is judged by the relevant body not possible: (a) for decisions on procedural issues, these shall be decided according to th e General Regulations of the WMO; (b) for approval, adoption and acceptance of reports, differing views shall be explained and,upon request, recorded. Differing views on matters of a scientific, technical or socioeconomic nature shall, as appropriate in th e context, be represented in the scientific, technical or socio-economic document concerned. Differences of views on matters of policy or procedure shall, as appropriate in the context, be recorded in the Report of the Session.

We disagree with the commenter that consensus was not reached amongst the governments and authors involved with the IPCC. Consistent with IPCC pro cedures, the Summary for Policymakers of Working Groups I, II, and III were ‘approved’ and the full repo rts of the three Working Groups were ‘accepted’. IPCC Procedures (IPCC, 1999) define ‘approval’ to mean that “the material has been subjected to detailed, line by line discussion and agreement,” and “acceptance” signifies that “the material has not been subject to line by line discussion and agreemen t, but nevertheless presents a comprehensive, objective and balanced view of the subject matter.” Finally, we note that most of the commenters provided no evidence to defend their arguments. The fe w pieces of evidence that were submitted focused on very narrow and specific findings of studies that were referenced by IPCC (2007). For EPA’s responses to these specific issues, please refer to the appropriate Response to Comment volumes.

Below, we provide excerpts of the relevant sections of the IPCC procedures (IPCC, 1999); the full text of the procedures is found in Appendix A.

Report development: To ensure objectivity, transparency, and information quality, the procedures (IPCC, 1999) clearly describe the roles and responsibiliti es of coordinating lead authors, lead authors, contributing authors, expert reviewers, review edito rs, and government reviewers in report development.

4.2.1 Compilation of Lists of Coordinating Lead Authors, Lead Authors, Contributing Authors, Expert Reviewers, Review Editors and Government Focal Points At the request of Working Group/Task For ce Bureau Co-Chairs through their respective Working Group /Task Force Bureau, and the IPCC Secretariat, governments, and participating organizations and the Working Group/Task Force Bureau should identify appropriate experts for each area in the Report who can act as potential Coordinating Lead Authors, Lead Authors, Contributing Authors, expert reviewers or Review Editors. To facilitate the identification of experts and later review by governments, governments should also designate their respective Focal Points. IPCC Bureau Members and Members of the Task Force Bureau should contribute where necessary to identifying appropriate Coordinating Lead Authors, Lead Authors, Contributing Authors, expert reviewers, and Review Editors in cooperation with the Govern ment Focal Points within their region to ensure an appropriate representation of e xperts from developing and developed countries



### 10and countries with economies in transition. Th ese should be assembled into lists available to all IPCC Members and maintained by the IPCC Secretariat. The tasks and responsibilities of Coordinating Lead Authors, Lead Authors, Contributing Authors, expert reviewers, Review Editors and governm ent Focal Points are outlined in Annex 1.

4.2.2 Selection of Lead Authors Coordinating Lead Authors and Lead Authors are selected by the relevant Working Group/Task Force Bureau, under general guidance and review provided by the Session of the Working Group or, in case of reports prepared by the Task Force on National Greenhouse Gas Inventories, the Panel, from t hose experts cited in the lists provided by governments and participating organizations , and other experts as appropriate, known through their publications and works. The composition of the group of Coordinating Lead Authors and Lead Authors for a section or chap ter of a Report shall reflect the need to aim for a range of views, expertise and ge ographical representation (ensuring appropriate representation of experts from developing a nd developed countries and countries with economies in transition). There should be at least one and normally two or more from developing countries. The Coordinating Lead Authors and Lead Authors selected by the Working Group/Task Force Bureau may enlist other experts as Contributing Authors to assist with the work.

At the earliest opportunity, the IPCC Secretariat should inform all governments and participating organizations who the Coordina ting Lead Authors and Lead Authors are for different chapters and indicate the general c ontent area that the person will contribute to the chapter.

4.2.3 Preparation of Draft Report Preparation of the first draft of a Report should be undertaken by Coordinating Lead Authors and Lead Authors. Experts who wish to contribute material for consideration in the first draft should submit it directly to the Lead Authors. Contributions should be supported as far as possible with references fro m the peer-reviewed and internationally available literature, and with copies of any un published material cited. Clear indications of how to access the latter should be included in the contributions. For material available in electronic format only, a hard copy shoul d be archived and the location where such material may be accessed should be cited.

Lead Authors will work on the basis of these contributions, the peer-reviewed and internationally-available literature, including manuscripts that can be made available for IPCC review and selected non-peer review literature according to Annex 2 and IPCC Supporting Material (see section 6). Material which is not published but which is available to experts and reviewers may be in cluded provided that its inclusion is fully justified in the context of the IPCC assessment process (see Annex 2).

In preparing the first draft, and at subsequent stages of revision after review, Lead Authors should clearly identify disparate view s for which there is significant scientific or technical support, together with the rele vant arguments. Technical summaries provided will be prepared under the leadership of the Working Group/Task Force Bureau.

Review: IPCC’s Procedures prescribe that “Review is an essential part of the IPCC process, to ensure an objective and complete assessment of current in formation.” The following excerpts from IPCC’s Procedures describe in detail the steps, requirements, and objectives of the review process.



### 114.1 Introduction to Review Process The review process generally takes place in three stages: expert review of IPCC Reports, government/expert review of IPCC Reports, government review of the Summaries for Policymakers, Overview Chapters and/or the Synthesis Report. Working Group/Task Force Bureau Co-Chairs should aim to avoid (or at least minimize) the overlap of government review periods for different IPCC Reports and with Sessions of the Conference of Parties of the United Nations Framework Convention of Climate Change and its subsidiary bodies.

Expert review should normally be eight weeks, but not less than six weeks, except to the extent decided by the Panel. Government and government/expert reviews should not be less than eight weeks, except to the extent decided by the Panel.

All written expert, and government review co mments will be made available to reviewers on request during the review process and will be retained in an open archive in a location determined by the IPCC Secretariat on completion of the Report for a period of at least five years....

...4.2.4 Review Three principles governing the review should be borne in mind. First, the best possible scientific and technical advice should be included so that the IPCC Reports represent the latest scientific, technical and socio-economic findings and are as comprehensive as possible.

Secondly, a wide circulation process, ensuring representation of independent experts (i.e. experts not involved in the preparation of that particular chapter) from developing and developed countries and countries with economi es in transition should aim to involve as many experts as possible in the IPCC process. Thirdly, the review process should be objective, open and transparent.

To help ensure that Reports provide a ba lanced and complete assessment of current information, each Working Group/Task Force Bureau should normally select two Review Editors per chapter (including the executive summaries) and per technical summary of each Report.

Review Editors should normally consist of a member of the Working Group/Task Force Bureau, and an independent expert based on the lists provided by governments and participating organizations. Review Editors s hould not be involved in the preparation or review of material for which they are an ed itor. In selecting Review Editors, the Bureau should select from developed and developing countries and from countries with economies in transition, and should aim fo r a balanced representation of scientific, technical, and socio-economic views.

4.2.4.1 First Review (by Experts) First draft Reports should be circulated by Working Group/Task Force Bureau Co-Chairs for review by experts selected by the Working Group/Task Force Bureau and, in addition, those on the lists provided by governments and participating organizations, noting the need to aim for a range of views, expertise, and geographical representation. The review circulation should include:

o Experts who have significant expertise and/or publications in particular areas covered by the Report.



### 12o Experts nominated by governments as Coordinating Lead Authors, Lead Authors, contributing authors or expert reviewers as included in lists maintained by the IPCC Secretariat.

o Expert reviewers nominated by appropriate organizations.

The first draft Reports should be sent to Gove rnment Focal Points, for information, along with a list of those to whom the Report h as been sent for review in that country.

The Working Group/Task Force Bureau Co-Cha irs should make available to reviewers on request during the review process specific material referenced in the document being reviewed, which is not available in the international pu blished literature.

Expert reviewers should provide the comments to the appropriate Lead Authors through the relevant Working Group/Task Force Bureau Co-Chairs with a copy, if required, to their Government Focal Point.

Coordinating Lead Authors, in consultation with the Review Editors and in coordination with the respective Working Group/Task Force Bureau Co-Chairs and the IPCC Secretariat, are encouraged to supplement the draft revision process by organizing a wider meeting with principal Contributing Authors and expert reviewers, if time and funding permit, in order to pay special atte ntion to particular points of assessment or areas of major differences.

4.2.4.2 Second Review (by Governments and Experts) A revised draft should be distributed by the appropriate Working Group/Task Force Bureau Co-chairs or through the IPCC Secretariat to governments through the designated Government Focal Points, and to all the c oordinating lead authors, lead authors and contributing authors and expert reviewers.

Governments should send one integrated set of comments for each Report to the appropriate Working Group/Task Force Bur eau Co-chairs through their Government Focal Points.

Non-government reviewers should send their further comments to the appropriate Working Group/Task Force Bureau Co-Chair s with a copy to their appropriate Government Focal Point.

Acceptance and Approval: Finally, IPCC’s Procedures (1999) describe the acceptance and approval procedures to ensure that the development of the assessment reports has been completed in a way which maximizes objectivity, transparency, quality, and utility:

4.2.5 Preparation of Final Draft Report Preparation of a final draft Report taking into account government and expert comments for submission to a Session of a Working Group or, in case of a report prepared by the Task Force on National Greenhouse Gas Invent ories, of the Panel for acceptance should be undertaken by Coordinating Lead Authors and Lead Authors in consultation with the Review Editors. If necessary, and timing and funding permitting, a wider meeting with principal Contributing Authors and expert a nd government reviewers is encouraged in order to pay special attention to particular points of assessment or areas of major differences. It is important that Reports describe different (possibly controversial) scientific, technical, and socio-economic views on a subject, particularly if they are



### 13relevant to the policy debate. The final draf t should credit all Coordinating Lead Authors, Lead Authors, Contributing Authors, re viewers and Review Editors by name and affiliation (at the end of the Report).

4.3 Approval and Acceptance of Summaries for Policymakers and Adoption of Overview Chapters of Methodology Report s Related to National Greenhouse Gas Inventories Summary sections of Reports approved by the Working Groups and accepted by the Panel will principally be the Summaries for Policymakers, prepared by the respective Working Groups of their full scientific, tec hnical and socio-econom ic assessments, and Summaries for Policymakers of Special Reports prepared by the Working Groups. The Summaries for Policy Makers should be subject to simultaneous review by both experts and governments and to a final line by line approval by a Session of the Working Group. Responsibility for preparing first drafts and revised drafts of Summaries for Policymakers, lies with the respective Work ing Group Co-Chairs. The Summaries for Policymakers should be prepared concurrently with the preparation of the main Reports.

Approval of the Summary for Policymakers at the Session of the Working Group, signifies that it is consistent with the factual material contained in the full scientific, technical and socioeconomic assessment or Special Report accepted by the Working Group. Coordinating lead authors may be asked to provide technical assistance in ensuring that consistency has been achie ved. These Summaries for Policymakers should be formally and prominently described as:

“A Report of [Working Group X of] the Intergovernmental Panel on Climate Change.”

For a Summary for Policymakers approved by a Working Group to be endorsed as an IPCC Report, it must be accepted at a Sessi on of the Panel. Because the Working Group approval process is open to all governments, Working Group approval of a Summary for Policymakers means that the Panel cannot change it. However, it is necessary for the Panel to review the Report at a Session, note any substantial disagreements, (in accordance with Principle 10 of the Princi ples Governing IPCC Work) and formally accept it.

As demonstrated by these procedures, IPCC assessment reports are developed following well-established and comprehensive guidelines. As an example of how this process works in practice, the introduction to IPCC (2007b), the working group report on impacts, adaptation and vulnerability, summarized their process as follows:

In total, the Working Group II Fourth Assessment involved 48 Coordinating Lead Authors (CLAs), 125 Lead Authors (LAs), a nd 45 Review Editors (REs), drawn from 70 countries. In addition, there were 183 Contri buting Authors and 910 Expert Reviewers.

Each chapter in the Working Group II Fourth Assessment had a writing team of two to four CLAs and six to nine LAs. Led by the CLAs, it was the responsibility of this writing team to produce the drafts and finished version of the chapter. Where necessary, they could recruit Contributing Authors to assist in their task. Three drafts of each chapter were written prior to the production of the fi nal version. Drafts were reviewed in two separate lines of review, by experts and by go vernments. It was the role of the REs (two



### 14to three per chapter) to ensure that the re view comments were properly addressed by the authors.

The authors and REs were selected by the Working Group II Bureau from the lists of experts nominated by governments. Due rega rd was paid to the need to balance the writing team with proper representation from developing and devel oped countries, and Economies in Transition. In the review by expe rts, chapters were sent out to experts, including all those nominated by governme nts but not yet included in the assessment, together with scientists and researchers id entified by the Working Group II Co-Chairs and Vice-Chairs from their knowledge of the literature and the global research community.

The evidence is clear that the IPCC’s procedures are sufficient and effective for ensuring quality, transparency, and consideration of multiple and dive rse perspectives. Because the assessment reports EPA used in developing the TSD represent the best av ailable science, and because supporting studies were conducted in accordance with sound an d objective scientific practices, were peer reviewed, and adhered to standards of quality based on objectivity, utility, an d integrity, we find that IPCC’s information quality process is consistent with EPA’s Guidelines for Ensuring and Maximizi ng the Quality, Objectivity, Utility and Integrity of Information Disseminated by the Environmental Protection Agency.

The U.S. Government participated fully in th e development, review, and ultimate acceptance and approval of IPCC (2007). As stated on the USGCRP ’s Web site: “When governments accept the IPCC reports and approve their Summary for Policymakers, they acknowledge the legitimacy of their scientific content” ( http://www.globalchange.gov/publi cations/r eports/ipcc-reports ).

Comment (1-15) : A commenter (3303) argues that the IPCC AR4 relied upon non-peer reviewed, unsupportable studies rather than peer-reviewed literature.

Response (1-15) : The IPCC Procedures (IPCC, 1999) used in preparing its assessm ent reports are described above and can be viewed in Appendix A of this volume. The IPCC provides a robust process for ensuring that the best available scientific information is used in thei r assessment report development process. The IPCC (2007) relied almost exclusively on peer-reviewed, publis hed information and we note that IPCC procedures require reliance on peer-reviewed literature except in specific instances. IPCC’s guidelines state that “Contributions should be supported as far as possibl e with references from the peer-reviewed and internationally available literature...” (IPCC, 1999). Limited unpublished material was included only if it was justified in the context of the IPCC assessment process and completely accessible.

Annex 2 of the IPCC procedures (IPCC, 1999) provid es the following guidelines for using non-published sources in assessment reports:

Because it is increasingly apparent that materi als relevant to IPCC Reports, in particular, information about the experience and practice of the private sector in mitigation and adaptation activities, are found in sources that have not been published or peer-reviewed (e.g., industry journals, internal organizational publications, non-peer reviewed reports or working papers of research institutions, pr oceedings of workshops etc) the following additional procedures are provided. These have been designed to make all references used in IPCC Reports easily accessible and to ensure that the IPCC process remains open and transparent.



### 151. Responsibilities of Coordinating, Lead and Contributing Authors Authors who wish to include informa tion from a non-published/non-peer-reviewed source are requested to: a. Critically assess any source that they wish to include. This option may be used for instance to obtain case study materials from private sector sources for assessment of adaptation and mitigation options. Each ch apter team should review the quality and validity of each source before incorporating results from the source into an IPCC Report. b. Send the following materials to the Working Group/Task Force Bureau Co-Chairs who are coordinating the Report: - One copy of each unpublished source to be used in the IPCC Report - The following information for each source: - Title - Author(s) - Name of journal or other publication in which it appears, if applicable - Information on the availability of underlying data to the public- English-language executive summary or abstract, if the source is written in a nonEnglish language - Names and contact information for 1-2 people who can be contacted for more information about the source. 2. Responsibilities of the Review Editors The Review Editors will ensure that these sources are selected and used in a consistent manner across the Report.

These procedures establish that the IPCC rely almost exclusively on peer-reviewed literature. The procedures also provide a comprehensive, transparen t and robust process in the rare circumstances where unpublished or non-peer reviewed material is used. Fu rther, EPA’s review of the studies included in the IPCC assessment reports, which we used in developing the TSD, confirms that the use of non-peerreviewed literature was predominantly associat ed with the report of IPCC Working Group III (“Mitigation of Climate Change”) and the treatment of adaptation issues in IPCC Working Group II, two areas not assessed by the Administrator in this action. See Section III.C, “Adaptation and Mitigation,” of the Findings for a discussion on the treatment of adaptation and mitigation.

The commenter describes two pieces of literature that were referenced by IPCC but not published in peerreviewed journals: 1) a natural catastrophes report by Munich Re (2000) that was cited in IPCC Third Assessment Report (IPCC, 2001), and 2) a paper on tr ends in weather-related catastrophes by Muir-Wood et al. (2006). The identification of only two examples out of the thousands of references cited in the IPCC (2000) and (2007) reports provides more support for the rigor of the IPCC process. Further, these specific studies were neither the central nor sole evidence used in forming the broader conclusions of the IPCC; such broader conclusions are based on multiple lines of evidence and peer-reviewed literature.

Comment (1-16) : Several commenters (e.g., 3680.1, 4101, 4143, 7 392, 8318, 8874, 10156) voice their support for EPA’s reliance on recently published, consensus-based, pee r-reviewed assessments and reports of the IPCC, CCSP, and the National Academy of Sciences. So me commenters (9568, 9633) provided additional supportive references, citing Dr. James Hansen, Lord Nicholas Stern, Rajendra Pachauri, James Lovelock, and Stephen Hawking. One commenter (9749) referenced a report from the New Jersey Board of Public Utilities on climate and energy detailing the impacts of climate change on the quality of life in New Jersey.



### 16Response (1-16) : We agree that reliance on the peer-re viewed assessment literature is appr opriate and reasonable. We have closely reviewed and considered the additional stud ies submitted by the commenter and we find that these materials are consistent with the information summarized in the TSD. For example, the New Jersey report states that “Not only does climate change threat en New Jersey’s shoreline and ecology, but the socioeconomic impacts of global warming stand to be profound and costly.”

Comment (1-17) : Several commenters (1005.1, 3303, 3330.1, 3427.1, 3446.3, and 3567.1) argue that many of IPCC’s conclusions are distorted, misleading, or incorrect . A number of commenters (0487, 1927, 3013, 3219.1, 3303, 3567.1, 3707.1, 3764.1, and 4509) state that IPCC reports are based on poor-quality science.

Response (1-17) : See Section III.A. of the Findings, “The Science on Wh ich the Decisions Are Based,” for our response to comments on the use of the assessment literature and previous responses in this section regarding our treatment of new and additional scientific literature provided through the public comment process. For EPA’s responses to comments and literature provide d on specific climate science issues in the TSD, please refer to the appropriate Response to Comment volumes.

We note that this response addresses the broad assertion made by commenters with regard to IPCC’s conclusions being distorted, misleading, or incorrect. Several of these commenters also provided specific reasons for why these conclusions are claimed to be distorted, misleading, or incorrect (e.g., due to inappropriate review processes) and we respond to these specific comments in the relevant volumes of this Response to Comments document.

As a general matter, we disagree that the IPCC’s conclusi ons are distorted, misleading, or incorrect or that the IPCC relies on poor-quality science. As thoroughly discussed in our response to comment 1-14, the IPCC process for preparation and review of its reports demonstrate a formal and sustained commitment to transparency and rigor in report development, review, and acceptance and approval. The assessment reports we used in developing the TSD represent the best available science and supporting studies conducted in accordance with sound an d objective scientific practices, are peer reviewed, and adhere to standards of quality based on objectiv ity, utility, and integrity.

Comment (1-18) : Several commenters (2679.1, 2972.1, 3136.1, 3323.1, 3327.1, 3330.1, 3347.4, 3394.1, 3794.1, 4003, 4041.1, 4932.1, 5158, and 11046) argue that the IPCC reports are inappropriate for use in the endangerment finding because the underlying studies are out of date and that major scientific findings have appeared since IPCC (2007) that have change d the science community's understanding of what is presented in that report.

Response (1-18) : See Section III.A. of the Findings, “The Science on Wh ich the Decisions Are Based,” for our response to comments on the use of the assessment literature and previous responses in this section regarding our treatment of new and additional scientific literature provided through the public comment process.

As part of this final Action, EPA updated the TSD to reflect new assessment reports published since release of the Proposed Findings and the TSD’s April 2009 version. In particular, we have revised the TSD to incorporate the findings of the latest assess ment (released June 16, 2009) of the USGCRP. This



### 17new study does not differ markedly from the fundamental conclusions of the IPCC. In fact, this most recent USGCRP assessment report integrates newer lite rature and builds on the key findings from the IPCC, USGCRP, CCSP, and NRC assessme nts to reach similar, and in some cases, stronger conclusions. In addition, the TSD incorporates up-to-date observa tional data for a number of key climate variables from the National Oceanic and Atmospheric Administ ration (NOAA), and the most up-to-date emissions data from EPA’s annual Inventory of U.S. Greenhouse Gas Emissions and Sinks , published in April 2009. We do not agree that it is inappropriate for the Admi nistrator to use the IPCC assessment reports as part of the foundation for the Administrator’s endangerm ent determination. The assessment reports discussed in the TSD do reflect the state of science on climate change and are not out of date. EPA has determined that the approach taken provided the high level of transparency and consistency outlined by EPA’s Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Inte grity of Information Disseminated by the Environmental Protection Agency.

EPA’s responses to comments and literature submitted on specific climate science issues in the TSD are provided in the appropriate Response to Comme nt volumes. These responses indicate where new scientific information (from either new assessment reports or individual studies) reinforce, refine, or correct specific issues. EPA has addressed all signi ficant scientific findings identified and submitted by commenters that have appeared since the IPCC (2007).

Based on our review of the studies submitted durin g the public comment process as well as additional peer-reviewed literature published since IPCC’s Fo urth Assessment Report, we find that IPCC’s conclusions are still accurate representations of the scientific community’s understanding of the science. Much of the more recent literature reaffirms IPCC’s conclusions, and even stre ngthens the confidence in key issues.

Comment (1-19) : Several commenters (3187.5, 3440.1, 4288, 9636, 9970, 11288, 11315, and 11409) argue that IPCC reports, including the Summary for Policymakers (SPM), are not appropriate resources for the endangerment finding because IPCC tasks government politicians with creating the documents instead of climate scientists.

Response (1-19) : We refer the commenters to previous responses to comments in this section regarding IPCC’s report development process.

EPA disagrees with this comment for several reasons. First, the IPCC has clear procedures in place for the preparation of the SPM, which state that: “Summa ry sections of Reports approved by the Working Groups and accepted by the Panel will principally be the Summaries for Policymakers, prepared by the respective Working Groups of their full scientific , technical and socio-economic assessments, and Summaries for Policymakers of Special Reports pr epared by the Working Groups” (IPCC, 1999). These procedures require the SPM to be drafted by teams of authors from the various underlying IPCC chapters, and the commenter has provided no information that demonstrates that “gove rnment politicians” were involved in the drafting.

Second, the IPCC procedures build in processes to ensure that the findings of the SPM are consistent with the underlying IPCC reports: “Approval of the Summary for Policymakers at the Session of the Working Group, signifies that it is consistent with the factual material contained in the full scientific, technical and socioeconomic assessment or Special Report accepted by the Working Group. Coordinating lead authors may be asked to provide technical assistance in en suring that consistency has been achieved” (IPCC,



### 181999). These procedures avoid the apparent concern of the commenter that an SPM could be inconsistent with the findings of the body of an IPCC report.

Third, in developing the TSD as the scientific basis for the Administrator’s endangerment analysis, EPA relied predominantly on the full reports of IPCC (2007a, 2007e), not on the SPMs. Findings from the SPMs were only used to provide overarching discu ssions of climate science and impacts throughout the TSD Sections.

Fourth, IPCC procedures and principles (see our resp onse to comment 1-14 and Appendix A) demonstrate that in making decisions and appr oving, adopting, and accepting reports , the Panel, its Working Groups and any Task Forces shall use all best endeavors to reach consensus. In t hose cases where consensus is not reached, IPCC procedures require that the diffe ring views shall be explained and, upon request, recorded. These procedures also specify that differing views on matters of a scientific, technical, or socioeconomic nature shall, as appropriate in the context, be represented in the scientific, technical, or socioeconomic document concerned. We note that neith er SPM for IPCC (2007a) or IPCC (2007e) indicate that consensus was not reached.

Any statement from an IPCC SPM used within the TS D is fully supported by the underlying, full report of the IPCC; all of the material taken from SPM Sec tions of these reports is supported by information in the underlying chapters. Commenters also have not identified any such stat ement that is inconsistent with this.

Comment (1-20) : Several commenters (1961, 3053, 3707.1, 3722, 3722.9, 3187.5, 7029, and 10196) argue that the authors involved in developing and reviewing IPCC (2007) all share the same views regarding climate change science, and that alternative perspectives were not incorporated into the pro cess. These commenters also state that the IPCC authors commonly collaborate on research, and therefore were not capable of objectively reviewing and editing the content of IPCC (2007). A commenter (3187.5) specifically mentioned that IPCC’s authors on the climate mode ling chapters do not represent a diversity of perspectives, as many of them work at the same res earch institutes and/or colla borate with each other.

Response (1-20) : We refer the commenters to previous responses to comme nts in this section (1-14, 1-15, 1-17, 1-18, 1-19) regarding IPCC’s report development process.

EPA has carefully reviewed the IPCC procedures and does not agree with this comment. For an overview of the IPCC procedures, please see the Introduction of the TSD. As noted previously, Section 1(b) and Box 1.1 of the TSD provide a general description of th e IPCC’s procedures. Both within the United States and within the IPCC, the objective was to ensure that the authors represented a range of perspectives and views. We note that the majority of these comment s provide no specific evidence for their general assertion. Only one commenter (3187.5) provided a specific argument—a critique of the range of perspectives among the authors on Chapter 9 (“Unders tanding and Attributing Climate Change”) of WGI (IPCC, 2007a). We acknowledge that some of the 53 Chapter 9 authors are associated with the same research institutes; however, we find that the majority of them do not frequently collaborate on research. In fact, many of the research institutes, such as National Center for Atmospheric Research and the Hadley Center, work on their own climate modeling studies i ndependently of other programs. Further, we find that the authors from Chapter 9 come from 31 di fferent organizations and from over 15 different countries. In a field as small and technical as climat e modeling, we conclude that representation from 31 organizations sufficiently provides a range of pers pectives. Even the commenter acknowledges this in their statement that “53 authors of chapter 9 came from just 31 organizations.”



### 19The commenters also claim that the IPCC authors co mmonly collaborate on research and therefore are not objective. This claim is not supported by the eviden ce. Each IPCC report is developed by a team of authors, broken down into independent groups focusing on specific chapters, under the leadership of the Working Group Chairs and Convening Lead Aut hors. Please see the excerpt from IPCC’s Procedures (IPCC, 1999) at the end of this comment, which discu sses the tasks and responsibilities for lead authors, coordinating lead authors, contributing authors, expert reviewers, and government reviewers. In addition to what is stated in this excerpt, the Procedures (IPCC, 1999) state that:

Coordinating Lead Authors and Lead Authors are selected by the relevant Working Group/Task Force Bureau, under general guidance and review provided by the Session of the Working Group....The composition of the group of Coordinating Lead Authors and Lead Authors for a section or chapter of a Report shall reflect the need to aim for a range of views, expertise and geographical represen tation (ensuring appropriate representation of experts from developing a nd developed countries and countries with economies in transition). There should be at least one and normally two or more from developing countries.

More than 500 authors and 2,000 reviewers (repr esenting over 100 nations) were involved with developing IPCC (2007), and they offered a diversity of perspectives related to climate change science both during the drafting of the reports and during multip le comment opportunities. We find that the level of collaboration amongst the hundreds of authors is low and that the examples of collaborative efforts (e.g., co-authorship of studies) provided by commenter s are isolated cases—especially when considering the large number of available studies upon which the assessment reports rely and the hundreds of authors who developed them. Further, there is no evidence provided that supports the claim that collaboration among a small number of the authors prevented the incorporation of a range of perspectives and alternative views or biased the conclusions.

In addition, Review Editors oversee the integrity of the response to comment processes for all IPCC reports. IPCC’s guidelines for report development (IPCC, 1999) state that:

Three principles governing the review should be borne in mind. First, the best possible scientific and technical advice should be included so that the IPCC Reports represent the latest scientific, technical and socio-economic findings and are as comprehensive as possible. Secondly, a wide circulation process, ensuring representation of independent experts (i.e. experts not involved in the pr eparation of that particular chapter) from developing and deve loped countries and countries with economies in transition should aim to involve as many experts as possible in the IPCC process. Thirdly, the review process should be objective, open and transparent.

To help ensure that Reports provide a ba lanced and complete assessment of current information, each Working Group/Task Force Bureau should normally select two Review Editors per chapter (including the executive summaries) and per technical summary of each Report.

Review Editors should normally consist of a member of the Working Group/Task Force Bureau, and an independent expert based on the lists provided by governments and participating organizations. Review Editors s hould not be involved in the preparation or review of material for which they are an ed itor. In selecting Review Editors, the Bureau should select from developed and developing countries and from countries with



### 20economies in transition, and should aim fo r a balanced representation of scientific, technical, and socio-economic views.

We do not find merit in the argument that the IPCC procedures or the IPCC authors lack objectivity or integrity. Please see the relevant Response to Comment volumes for our responses to any specific technical comments provided on the IPCC findings or processes.

The following is an excerpt from IPCC’s Procedures (IPCC, 1999) describing the tasks and responsibilities for lead authors, coordinating lead authors, contributing authors, expert reviewers, and government reviewers. Appendix A of this volume contains IPCC’s Procedures (IPCC, 1999) in its entirety.

1. LEAD AUTHORS Function: To be responsible for the production of desi gnated sections addressing items of the work program on the basis of the best scientific, technical and socio-economic information available.

Comment: Lead Authors will typically work as small gr oups which have res ponsibility for ensuring that the various components of their sections are brought together on time, are of uniformly high quality and conform to any overall standards of style set for the document as a whole.

The task of Lead Authors is a demanding one and in recognition of this the names of Lead Authors will appear prominently in the final Report. During the final stages of Report preparation, when the workload is often particularly heavy and when Lead Authors are heavily dependent upon each other to read and edit material, and to agree to changes promptly, it is essential that the wo rk should be accorded the highest priority.

The essence of the Lead Authors’ task is synthesis of material drawn from available literature as defined in Section 4.2.3. Lead Authors, in conjunction with Review Editors, are also required to take account of expe rt and government review comments when revising text. Lead Authors may not necessarily write original text themselves, but they must have the proven ability to develop text that is scientifically, technically and socioeconomically sound and that faithfully represents , to the extent that this is possible, contributions by a wide variety of experts. The ability to work to deadlines is also a necessary practical requirement. Lead Authors ar e required to record in the Report views which cannot be reconciled with a c onsensus view but which are nonetheless scientifically or technically valid.

Lead Authors may convene meetings with C ontributing Authors, as appropriate, in the preparations of their sections or to discuss expert or government review comments and to suggest any workshops or expert meetings in their relevant areas to the Working Group/Task Force Bureau Co-Chairs. Th e names of all Lead Authors will be acknowledged in the Reports.

2. COORDINATING LEAD AUTHORS Function: To take overall responsibility for coordinating major sections of a Report



### 21Comment: Coordinating Lead Authors will be Lead Authors with the added responsibility of ensuring that major sections of the Report are completed to a high standard, are collated and delivered to the Working Group/Task Force Bureau Co-Chairs in a timely manner and conform to any overall standards of style set for the document.

Coordinating Lead Authors will play a lead ing role in ensuring that any crosscutting scientific or technical issues which may involve several sections of a Report are addressed in a complete and coherent manner a nd reflect the latest information available.

The skills and resources required of Coordinating Lead Authors are those required of Lead Authors with the additional organizationa l skills needed to coordinate a section of a Report.

The names of all Coordinating Lead Authors will be acknowledged in the Reports.

3. CONTRIBUTING AUTHORS Function: To prepare technical information in the form of text, graphs or data for assimilation by the Lead Authors into the draft section.

Comment: Input from a wide range of contributors is a key element in the success of IPCC assessments, and the names of all contributors will be acknowledged in the Reports. Contributions are sometimes solicited by Lead Authors but unprompted contributions are encouraged.

Contributions should be supported as far as possible with references from the peer reviewed and internationally available lite rature, and with copi es of any unpublished material cited; clear indications of how to access the latter should be included in the contributions. For material available in elect ronic format only, the location where such material may be accessed should be cited.

Contributed material may be edited, merged and if necessary, amended, in the course of developing the overall draft text.

4. EXPERT REVIEWERS Function: To comment on the accuracy and completeness of the scientific/technical/socio-economic content and the overall scientific/technical/s ocio-economic balance of the drafts.

Comment: Expert reviewers will comment on the text according to their own knowledge and experience. They may be nominated by Governments, national and international organizations, Working Group/Task Force Bureau/Lead Authors and Contributing Authors.

5. REVIEW EDITORS Function: Review Editors will assist the Working Gr oup/Task Force Bureau in identifying reviewers for the expert review process, ensure that all substantive expert and



### 22government review comments are afforded appropriate consideration, advise lead authors on how to handle contentious/controversial i ssues and ensure genuine controversies are reflected adequately in the text of the Report.

Comment: There will be one or two Review Editors per chapter (including their executive summaries) and per technical summary. In order to carry out these tasks, Review Editors will need to have a broad understanding of th e wider scientific and technical issues being addressed. The workload will be particularly heavy during the final stages of the Report preparation. This includes attending those m eetings where writing teams are considering the results of the two review rounds. Review Editors are not actively engaged in drafting Reports and cannot serve as reviewers of those chapters of which they are Authors. Review Editors can be members of a Working Group/Task Force Bureau or outside experts agreed by the Worki ng Group/Task Force Bureau.

Although responsibility for the final text remains with the Lead Authors, Review Editors will need to ensure that where significan t differences of opinion on scientific issues remain, such differences are d escribed in an annex to the Report. Review Editors must submit a written report to the Working Group Sessions or the Panel and where appropriate, will be requested to attend Sessions of the Working Group and of the IPCC to communicate their findings from the review process and to assist in finalizing the Summary for Policymakers, Overview Chapte rs of Methodology Reports and Synthesis Reports. The names of all Review Edito rs will be acknowledged in the Reports.

6. GOVERNMENT FOCAL POINTS Function: To prepare and update the list of national e xperts as required to help implement the IPCC work program, and to arrange the provision of integrated comments on the accuracy and completeness of the scientific and/or technical content and the overall scientific and/or technical balance of the drafts.

Comment: Government review will typically be carri ed out within and between a number of Departments and Ministries.

For administrative convenience, each govern ment and participati ng organization should designate one Focal Point for all IPCC activiti es, provide full information on this Focal Point to the IPCC Secretariat and notify th e Secretariat of any changes in this information. The Focal Point should liaise with the IPCC Secretariat regarding the logistics of the review process(es). Of particular importance is the full exchange of information.

Comment (1-21) : Several commenters (0800.1, 3440.1, 4528, 4666, 4670, and 4766) argue that IPCC relies too heavily on models and simulation tools in maki ng conclusions and that climate mode ls are inappropriate tools for an endangerment analysis.

Response (1-21) : EPA received many specific comments on the use of models and simulation tools in the context of the TSD and as support for the endangerment analysis . Please see the relevant Response to Comment



### 23volumes for our responses to these specific comments, particularly Volume 4, which focuses specifically on climate models and highlights other lines of evidence relevant to particular conclusions.

As a general matter, we find that IPCC’s reliance on models to evaluate the key elements of climate change science is appropriate and that these tools help scientists better understand the many climatic, geological, chemical, and biological systems involved with climate change. Furthermore, we find that the IPCC carefully documents and discusses uncertainties, assumptions, and limitations of the models used, and we have incorporated this information into the TS D. The climate models and other models used in the IPCC report meet the standards of the best available science available: they have been validated and compared against theory, observation, and other mode ls. The models are routinely used in scientific practice throughout the country and the world. The TSD appropriately represents both the strengths and limitations of the models, as determined by the major assessment reports.

In addition, it is important to recognize that the mo st important conclusions regarding climate science and impacts are based on numerous independent sources of information, including observations of the changing GHG composition of the atmosphere, observe d changes in the climate system, and observed changes to physical and biological systems.

Comment (1-22) : Several commenters (3394.1, 3440.1, 3722, and 4509) specifically state that in developing the TSD, EPA did not give proper consideration to the numerous uncertainties in IPCC's findings.

Response (1-22) : Please see Section 1.2 of this document for responses to these comments and for a general discussion of uncertainty. In addition, please see the relevant R esponse to Comment volumes for our responses on the treatment of uncertainty regarding specific issues in the TSD.

Comment (1-23) : Several commenters (3394.1, 3705.1, 3747.1, and 3482.1) argue that IPCC assessments are devoted primarily to global effects and global emissions and are therefore inappropriate for describing the observed and projected impacts of climate change in the United States necessary for an endangerment finding.

Response (1-23) : We do not agree that IPCC reports are inappropriate for an assessment of climate impacts in the United States. As explained in the TSD, the IPCC (2007b) includes a chapter on North America, which focuses on impacts, adaptation and vulnerability in the United States and Canada. We find that the North America chapter draws from many individual studies on th e United States and provides United States–specific findings, many of which are included in the TSD. We also note that IPCC (2007) contains a chapter on small islands with specific findings for U.S. islands and territories, along with a chapter on polar regions which includes Alaska-specific discussions. Where th e TSD references IPCC conclusions for global or regional areas that include the United States and other areas, we ensured that the conclusions were applied appropriately and accurately. Perhaps more importantly, we ensured that the TSD did not summarize conclusions from the assessment literature which did not apply to or were not appropriate for the United States. We received comments of this nature regarding specific climate science issues and our responses can be seen in the relevant volumes of this Respon se to Comments document. Our approach is entirely reasonable and consistent with how IPCC in tended the conclusions to be used.



### 24Finally, we note that the TSD summarizes importa nt findings and conclusions from reports of the USGCRP and CCSP, which focus almost exclusively on the United States. In particular, we updated the TSD to reflect the findings of the June 2009 USGCRP assessment report, by adding additional information to the sectoral chapters in Section IV of the TSD and a new section (15) on U.S. regional climate change impacts.

Please see other relevant Response to Comment volumes for our response to specific comments related to the TSD’s discussion of particular U.S. climate im pacts, and Volume 9 for our response to comments on the global and domestic scope of the endangerment finding.

Comment (1-24) : Several commenters (3287.1, 3291.1, 3379.1, 3394.1, 3407, 3482.1, and 3705.1) state that EPA’s adoption of the IPCC’s regional analysis of North Amer ican impacts, including projections for changes in temperature, precipitation, and sea level rise, cannot substitute for a U.S.–specific analysis because the IPCC’s assessment failed to distinguish adequately between U.S. and Canadian effects.

Response (1-24) : Please refer to the previous comment for our respons e regarding the applicability of the IPCC discussion of climate impacts to the United States.

1.1.2 Specific Comments on EPA’s Use of CCSP and USGCRP Reports

Comment (1-25) : A commenter (3765.1) encourages EPA to refer to the findings of the USGCRP's assessment report Global Climate Change Impacts in the United States.

Response (1-25) : The TSD has been revised to cite the final version of this report (Karl et al., 2009).

We find that this report and the other scientific assessments of the IPCC, CCSP, and NRC represent the best reference materials for dete rmining the general state of knowledge on the scientific and technical issues before the agency in informing an endangerment decision. The Guidelines for Producing CCSP Synthesis and Assessment Products (CCSP, 2004) state that:

I ndependent scientific judgment should serve as the guiding force in preparing the products so they are credible.
S cientists, users, and other stak eholders should jointly determine the scope of the products so the topics covered are well defined and the informati on provided is relevant to the needs expressed.
T he process of preparing the products should be open at every step so the products have legitimacy (i.e., are perceived to have been prepared fairly).

The process used by USGCRP/CCSP in developing thei r assessment reports is robust, transparent, and objective. We refer the commenters to Appendix B of this volume, which contains the following CCSP documents: 1) Guidelines for Producing CCSP Synthesis and Assessment Products (2004), 2) a CCSP

memo on Clarification of Review and Clearance Pro cess for CCSP Synthesis and Assessment Products

, (2007), and 3) Guidance to Agency Leads Regarding the Pre paration of CCSP Synthesis and Assessment



### 25Products (2006). Regarding the steps necessary in de veloping the assessment reports, CCSP (2004) describes the following procedures:

Planning the Process and Preparing a Prospectus 1. The lead agency(ies) solicit input from users and other stakeholders, plan preparation of the product, and summarize the proposed process in a draft prospectus. The draft prospectus will address the topics listed in th e subsequent section of this document.

2. The CCSP Interagency Committee reviews and approves the draft prospectus for public comment.

3. Expert reviewers and stakeholders revi ew the draft prospectus. The prospectus comment period will last at least 30 days. Th e draft prospectus comment period will be announced in a Federal Register Notice (FRN) and posted on the CCSP web site.

4. The lead agency(ies) revise the draft pr ospectus and finalize author recommendations, taking into consideration the comments received.

5. The CCSP Interagency Committee approves the revised prospectus and the lead agency(ies) notify the lead authors.

6. The CCSP Office posts the draft prospectus comments and the final prospectus on the CCSP web site.

Additional Stakeholder Interactions, if Needed 7. Lead authors may solicit additional input fro m users and other stakeholders to assist in the development of the product. The process for soliciting this additional input will be open and described in the prospectus. Appr oaches include workshops, user surveys, telephone and email conferences, and other m echanisms. The processe s used will reflect the expected end use of the product. The CC SP Strategic Plan identifies three end uses for CCSP synthesis and assessment products: 1) informing the evolution of the research agenda; 2) supporting adaptive management and planning; and 3) supporting policy formulation. The products with end uses prim arily oriented toward the second and third categories are expected to require significan t additional input from users to develop a clear understanding of information needs, timing of decisions, consideration of how uncertainty affects decision making, and other issues. The results from additional stakeholder interactions will be publicly av ailable in summary or more extensive forms through publication on the CCSP web site.

Drafting/Reviewing the Products 8. Lead authors prepare the first draft, including a technical section and a summary for interested non-specialists.

9. The lead agency(ies) organize and facilitate an expert peer review of the first draft according to process described in the prospectus. The expert peer review will precede the public comment period to ensure that the products are shaped by scientific considerations. The expert peer review pro cess may range from that used in a scientific journal to a formal review panel convened by the lead agency or recognized external groups such as the National Research C ouncil. Participants must be qualified scientific/technical experts, as demonstrated by their record of scholarly publication and other accomplishments. Employees of the lead agency(ies), lead authors, and other



### 26contributors to the product may not serve as expert reviewers for that product. The prospectus will describe the process for selecti ng expert reviewers and the expected dates of the review. If the expert peer review is open to all qualified experts, notice will be disseminated on the CCSP web site and through relevant scientific publications, web sites, and other means. All comments submitte d during the expert peer review will be publicly available without attribution to the reviewer unless reviewers agree in advance to posting with specific attribution.

10. Lead authors prepare the second draft of the product, taking into consideration the expert peer review comments. The scientific judgment of the lead authors will determine responses to the comments. The authors will acknowledge significant contributions made by expert reviewers, as applicable.

11. The lead agency(ies) post the second draf t of the product for public comment for not less than 45 days. Any stakeholders (plus expe rts who participated in the expert peer review process) may participate in the public comment period for the second draft. This includes governmental and non-governmental entities. The prospectus will include the expected dates of the public comment period. Notice of the public comment period will be disseminated on the CCSP web site, in the Federal Register, and through other publications, web sites, and means as appropriate to the product, to encourage wide public participation in the review. All comments will be publicly available.

12. The lead authors will prepare a third draft of the product, taking into consideration the comments submitted during the public comment period. The scientific judgment of the lead authors will determine responses to the comments.

Approving, Producing, an d Releasing the Products 13. Lead agency(ies) submit the third draft of the product and a compilation of comments received to the CCSP Interagency Committee.

14. If the CCSP Interagency Committee review determines that no further action is needed and that the product has been prepared in conformance with these guidelines and the Data Quality Act (including ensuring objectiv ity, utility, and integrity as defined in 67 FR 8452), they will submit the product to N STC for approval. If the CCSP Interagency Committee determines that further revision is necessary, their comme nts will be sent to the lead agency(ies) for considerati on and resolution by lead authors.

15. If needed, NRC can be asked to provid e additional scientific analysis to bound scientific uncertainty associated with specific issues.

16. Once the CCSP Interagency Committee h as determined that the synthesis and assessment report has been prepared in c onformance with these guidelines and the Data Quality Act, the Committee will submit it to NSTC for final review and approval. Approval will require the concurrence of a ll members of the Committee on Environment and Natural Resources. Comments generated during the NSTC review will be addressed by the CCSP Interagency Committee.

17. Once NSTC approval has been obtained and the product is finalized, the lead agency(ies) will produce and release the comp leted product using a standard format for all CCSP synthesis and assessment produc ts. The final product and the comments



### 27received during the expert review and the pub lic comment period will be posted, without attribution (unless specific reviewers agree to attribution), on the CCSP web site.

18. The product will be widely dissemina ted through the CCSP web site and other mechanisms.

The USGCRP/CCSP Guidelines (CCSP, 2004) lay out specific procedures regarding the roles and responsibilities of the lead agency or agencies, lead and contributing authors, in teragency working groups, expert reviewers, stakeholders, NRC, and th e National Science and Technology Council. These procedures were closely adhered to in developing th e assessment reports and we note that no commenters submitted evidence to the contrary. The implementati on of these guidelines ensured that the reports were being written using the best available science and objectively peer-reviewed. The Guidelines state:

Lead Agency(ies)/Department(s) One or more designated CCSP agency(ies) or department(s) will take the lead in producing each product. The lead agency(ies) w ill be responsible for developing an open and transparent process for soliciting user input, author nomination and selection, expert peer review and public comment, and produc tion/release of the products, as described in these guidelines. To ensure that the products incorporate as much expertise as possible, the lead agency(ies) will be open to the partic ipation of other individuals or entities with relevant expertise and information. The entities can include other government units (Federal or non-Federal), Interagency Work ing Groups of the CCSP or other Federal programs, international organizations and government units, non-governmental organizations (NGOs), and other groups.

Lead and Contributing Authors Lead and contributing authors of the synthesi s and assessment products are scientists or individuals with recognized technical exper tise appropriate to a product. Lead and contributing authors may be citizens of any c ountry and be drawn from within or outside the Federal government (e.g., universities or ot her public or private sector organizations). These individuals shall be acknowledged expert s, known through their publication record and relevant accomplishments and contributions to their field. Lead authors are responsible for the content of the synthesis and assessment products that are submitted to the CCSP Interagency Committee for review.

Interagency Working Groups The CCSP Interagency Committee coordinates im plementation of its activities in support of the Strategic Plan through Interagency Working Groups (IWGs) of program specialists of its participating departments and agencies, as described in Chapter 16, “Program Management and Review,” of the CCSP Strategic Plan. IWGs will contribute significantly to the preparation of the synth esis and assessment products because of their expertise in areas related to the products . IWGs may serve as a means for the lead agency(ies) to coordinate preparation of th e products with supporting agencies. They may contribute to planning/preparing the prospectuses, scoping, drafting, reviewing, publishing, or disseminating the final product.

Expert Reviewers Expert reviewers are scientists or individuals with other special expertise appropriate to a product. The expert reviewers will be selected by the lead agency(ies)/departments. As is the case for lead and contributing authors, re viewers may be citizens of any country and be drawn from within or outside the Federal government (e.g., universities or other public



### 28or private sector organizations). These individuals shall be known through their publications and other forms of recognition of their expertise. Expert reviewers will focus on the scientific/technical content of the draf t. Employees of the lead agency(ies), lead authors, and other contributors to the product may not serve as expert reviewers for that product. The expert reviewers will be desi gnated through a process described in the prospectus.

Stakeholders Stakeholders are defined as they are in Chapter 11 of the CCSP Strategic Plan—that is, “Stakeholders are individuals or groups whose interests (financial, cultural, value-based, or other) are affected by climate variability, climate change, or options for adapting to or mitigating these phenomena.” Stakeholders participate during the scoping process by providing information that helps define the audience and potential uses of a product. In addition, stakeholders provide comments on th e prospectus, and on the product during the public comment period. These comments are e xpected to focus on how well the product serves its intended purpose or use.

National Research Council The National Academy of Sciences / National Research Council (NRC) will provide advice on an as-needed basis to the lead agency(ies). In the ev ent that issues are identified that require further clarification, the NRC may be asked to provide additional scientific analyses to help bound the uncertain ty associated with these issues.

National Science and Technology Council The National Science and Technology Counc il (NSTC) will be responsible for final review and approval of the synthesis and assessment products. Products not cleared by NSTC cannot be released as disseminations of the Federal government. Consistent with NSTC procedures, approvals will require wr itten concurrence from all members of the NSTC’s Committee on Environment and Natural Resources (CENR). All comments generated through the NSTC review will be addressed by the CCSP Interagency Committee. The CENR membersh ip includes senior official s representing the Executive Office of the President and the 15 federal agencies with significant responsibilities for environment and natural resources programs.

The USGCRP/CCSP Guidelines (CCSP, 2004) also lay out specific procedures regarding the types of scientific materials to be used in the development of the assessment reports. This ensures that the reports are credible, robust, and representative of the best available science. After reviewing the final USGCRP and CCSP products, we find that these guidelin es were consistently implemented. The Guidelines state:

Authors will use the published, peer-review ed scientific literature in drafting the products. In the rare case that any materials used in preparing a product are not already published in the peer-reviewed literature, the lead agency(ies) must get approval from the CCSP Interagency Committee and these materials must be made available by the lead agency(ies) and/or CCSP Office. The use of any such non-peer-reviewed materials may be questioned by reviewers during the expert review or public comment period. Authors should seek to publish any materials used in preparing drafts of the products.

To ensure transparency in the assessment repor t development process, USGCRP/CCSP provided the public with access to a series of documents, includi ng invitations for public comment periods on the prospectus’ and preliminary drafts , compilations of comments received from the public and experts, and revised versions of the reports. For each assessment report, the following products were posted on the



### 29appropriate synthesis and assessment product (SAP) Web site, although we note that additional SAPspecific products were posted for many of the reports:

I nvitation for public comment on draft prospectus
D raft prospectus
P ublic comments on draft prospectus
F inal prospectus
P eer review plan (if not part of the prospectus)
E xpert review comments on first draft report and authors’ responses to them
I nvitation for public comment on second draft report
S econd draft report
P ublic comments on second draft report and authors’ responses to them
T hird draft report
F inal report

Finally, to ensure that each assessment report comp lied with the Information Quality Act (IQA) and was fully responsive to all comments received from th e public and expert review, the USGCRP/CCSP Guidelines (CCSP, 2004) state that:

When the lead agency transmits the product for final clearance, they should include a memorandum briefly indicating that “the product was prepared in compliance with CCSP’s Guidelines for Producing Synthesis and Assessment Products, the Information Quality Act (Section 515) and [LEAD AG ENCY’S] corresponding IQA guidelines; and the Federal Advisory Committee Act [when applicable].” This transmittal should include the authors’ responses to the peer reviewer comments, as required by OMB’s Information Quality Bulletin for Peer Review, as well as descriptions of how the authors addressed the public comments and lead agency’s review comments.

Comment (1-26) : A commenter (4036.5) states that EPA's use of CCSP 4.6 (Gamble, 2008) and the IPCC (2007) Working Group II chapter on human health impacts represent the consensus opinion of the world’s leading scientists studying health impacts of climate change.

Response (1-26) : CCSP 4.6 and the IPCC chapter on human health impacts are robust sources of information to serve as primary sources for the discussion of health impacts. See Volume 5 of the Response to Comments for responses to specific comments on human health and air quality.

Comment (1-27) : Two commenters (3446.2 and 3136.1) note numerous errors and contentious statements in the public draft of USGCRP's assessment report Global Climate Change Impacts in the United States and cautioned that this document was not based on best available science.

Response (1-27) : EPA did not cite the draft USGCRP report in the ve rsion of the TSD released in April 2009 with the Proposed Findings. However, we have revised the TSD to incorporate the c onclusions of the final USGCRP report, which was released in June 2009 (K arl et al., 2009). Regarding the quality and accuracy of the public review draft, we note that USGCRP substantially revised the report following the two



### 30separate rounds of public comment and expert re view. Commenters can review how USGCRP responded to the comments received at ( http://www.climatescience.gov/Library/sap/usp/default.php ).

Please see other relevant Response to Comment volumes for our response to specific comments on the USGCRP that relate to the TSD’s discussion of par ticular issues. See our response to other comments summarized in this section of this volume for a descr iption of our rationale as to why the USGCRP report represents the best available scientific information and, in its final form, meets the TSD's established reference criteria.

Comment (1-28) : As evidence why the EPA should not rely on CCSP synt hesis reports, a commenter (3136.1) submitted to EPA their comments on the USGCRP’s Global Climate Change Impacts on the United States, originally submitted during the public comment period for that re port. While nearly all of the comments raised by commenter 3136.1 address specific issues regarding the climate science and impacts information within the USGCRP report, one overarching issue raised in the comments was that the USGCRP authors made little effort to consider or include any dis senting opinion to their declarative statements.

Response (1-28) : EPA has reviewed the comments submitted, and our responses are provided in the relevant Response to Comment volumes related to the TSD’s discussion of pa rticular climate science and impacts issues. We also refer commenters to previous responses to co mments in this section regarding the USGCRP/CCSPreport development process. We also note that Box 1.1 of the TSD provides a description of the peerreview, publication, and approval processes employ ed by CCSP and USGCRP in developing their assessment reports.

The overarching comment that USGCRP made little effort to consider or include dissenting opinions is inconsistent with the process used by USGCRP in developing the report. We note that the USGCRP report and the previously developed CCSP reports have assessed numerous i ndividual, peer-reviewed studies in order to draw general conclusions about th e state of the science. This process involves weighing the quality of different individual studies (incl uding dissenting opinions), reconciling studies with divergent conclusions, and acknowledgi ng divergent conclusions (or a dearth of findings for a given area) to highlight key scientific uncertainties and gaps in knowledge. Further, CCSP and USGCRP provided multiple opportunities for input through several public comment periods, which were held for each SAP and the synthesis report. These comment periods a llowed the public to submit dissenting opinions and studies that were considered by CCSP prior to re leasing the final reports. Responses to the comments received during the two rounds of public review and two rounds of expert peer-review on the draft synthesis report can be viewed at http://www.climatescience.gov/Library /sap/us p /default.php . Finally, we note that the USGCRP/CCSP memorandum on Clarification of Review and Clearance Process for CCSP Synthesis and Assessment Products states that final clearance procedures employed by CCSP ensure that the officials with legal responsibility for the assessmen t reports have reviewed and approved them, with particular consideration that “the expert review and public comments were adequately addressed by the authors.”

Comment (1-29) : A commenter (2684.1) argues that the CCSP reports c ited by EPA are not used as references in peerreview scientific papers, and that without technical merit, these papers are of little scientific value.

Response (1-29) :



### 31We disagree with the commenter that the CCSP reports are not being used as references in reports and papers. We analyzed the citation rate of CCSP re ports in recent published, peer-reviewed studies and found that these assessments are being referenced fre quently, especially since these reports were only recently released (most in 2008 and 2009). For exam ple, a CCSP report on temperature trends in the lower atmosphere (CCSP, 2006) has been cited in over 50 papers published in peer-reviewed science journals. For the reasons provided in the Findings, Section III.A., and our response to comments in the section, we find that the CCSP reports represent scientif ically robust and credible sources of information on climate science and impacts.

Comment (1-30) : A commenter (3411.1) argues that there are additional studies on the issue of air quality impacts that the Agency does not address in the proposed endangerm ent finding or the TSD. The commenter requests an on-the-record proceeding at which all the available government-funded studies can be assessed.

Response (1-30) : See previous responses in this section for our response regarding the approach taken to review and reflect additional studies and new literature in the endange rment analysis. See Response to Comment Volume 5 for our response to specific comments on the issue of air quality impacts, which includes responses to additional studies submitted through the public comment process. See Section 1.C.1(e) of the Findings for our response to comments on the subject of an “on-the-record proceeding” under the Administrative Procedures Act.

1.2 Level of Scientific Certainty

Comment (1-31) : A commenter (2692) indicates that EPA’s discussion of the health impacts of climate change contains ambiguous terms such as “difficult to ascertain,” “exp ected,” “could,” “potential,” “likely,” “can,” “very likely,” “likely to,” “may,” and “projected.” Yet the EPA draws a “compelling and indeed, overwhelming” conclusion from such ambiguity. The commenter argues that how EPA arrives at this conclusion is puzzling.

Response (1-31) : In response to this comment, EPA has reviewed both the use of these terms in the TSD and Findings, as well as the explanation of our approach in the TSD. We agree with the commenter that clear and careful communication of uncertainty is important, and this is , in fact, a strength of the TSD. Box 1.2 of the TSD summarizes the approach and lexicon developed by the IPCC, USGCRP, and CCSP for the consistent communication of uncertainty and likelihood. As the TSD makes clear, the IPCC, USGCRP, and CCSP have established guidelines for the use of these types of terms precisely to minimize the ambiguity and misinterpretation that can occur when their meaning is not defined. In summarizing the existing and fully vetted literature, we adopted this existing approach.

Many of the terms identified by the commenter ar e defined in Box 1.2, to aid readers in better understanding the implications of various statemen ts throughout the TSD. For example, “likelihood” refers to a probabilistic assessment of some well-de fined outcome having occurred or occurring in the future, and may be based on quantitative analysis or an elicitation of expert views. Terms used to describe likelihood of certain outcomes include “virtually certa in,” “very likely,” “likely,” “about as likely as not,” “unlikely,” “very unlikely,” and “ex ceptionally unlikely”; as summarized in the Box 1.2, each term has an associated probability of occurrence. The term “projected” is used to explain an outcome in the future



### 32typically based on modeling analysis and are often accompanied by a term to communicate the outcome’s probability. Because the TSD summarizes the results of the major climate assessment reports, we concluded it was neither necessary nor appropriate for EPA to develop a new vernacular to convey likelihood and probability information. There are some phrases used in the TSD, such as “difficult to ascertain,” that indicate that available information is insufficient for experts to make an assessment or assign a probability.

In response to this comment, we also revised B ox 1.2 of the TSD to enumerate potential sources of uncertainty and add discussion about the importance of characterizing uncertainty. We also made revisions to more clearly describe how confid ence levels were developed in IPCC (2007).

See Section IV.B. of the Findings for a description of how the Administrator weighed the information contained in the TSD and broader record in making th e endangerment determinati on, and her view of the weight of the evidence.

Comment (1-32) : Some commenters (e.g., 1927) state that policy and re gulation should not be b ased on likelihood and that IPCC statements, for example, cannot be proven with certainty. A commenter (3397) argues that EPA did not provide sufficient guidance on the likelihood of occurrence of impacts.

Response (1-32) : See our response to the previous comment regardi ng how EPA communicated information on uncertainty and likelihood in the TSD. See II.A.2.a. of the Findings for our response on the question of whether policy and regulation can be based on likelihood, and S ection IV.B. of the Findings for our response to comments regarding how the Administrator weighed the information contained in the TSD and the broader record.

To the extent possible, EPA indicates the likelihood of occurrence in its discussion of impacts. Box 1.2 in EPA’s TSD describes the communication of confiden ce and uncertainty for IPCC and USGCRP (or CCSP) assessments, which are its primary underlyi ng references. When EPA cites these references, it retains their confidence and uncertainty language.

Comment (1-33) : A large number of commenters (e.g., 0463, 0727, 1927, 2253, 2750, 2898.1, 2923, 3080, 3347.1, 3384, 3427.1, 3435.1, 3498.1, 3507.1, 3559.1, 3577.1, 3559.1, 3577.1, 3702.2, 3830.1, 4738, 4173ANPRref1, 8978.1, A13, 10855, 10160, 11459) posit that the ev idence for human-induced warming is weak or the consequences of global warming, even if human-i nduced, are too uncertain to make an endangerment finding. Many commenters note the evidence of harm is equivocal and/or doubt that possible current/future negative effects from warming outwe igh benefits. Some commenters (4002, 3435.1) suggest that in light of the uncertainties enumerated by EPA and potential positive climate change effects, the endangerment finding appears to be more of a political than a scientific determination.

Response (1-33) : The Administrator carefully evaluated the scientific evidence summarized in the TSD, which reflects the major conclusions of the assessment literature on issu es pertinent to this finding. The public comments submitted on this action, including othe r scientific literature, were also reviewed and considered. On the basis of her review, the Administrator finds that the scientific information in totality provides compelling evidence of human-induced climate change. Section IV.B. of the Findings describes the Administrator’s



### 33determination regarding the consequences of climate change and why they justify a finding of endangerment. For the reasons explained there, the Administrator rejects the view that the consequences are too uncertain to justify an endangerment finding.

Many of the commenters who maintain there is too much uncertainty to support an endangerment finding raise specific scientific and technical concerns; our responses to these specific comments are provided in the other volumes of the Response to Comments document.

At least one commenter incorrectly claims the decision is more a political than a scientific determination. Section II of the Findings (“the Legal Framework for this Action”) explains how Section 202(a) requires the Administrator to exercise her judgment and apply the evidence to the statutory endangerment criteria. As stated there, “[b]Because scientific knowledge is constantly evolving, the Administrator may be called upon to make decisions while recognizing the uncertainties and limitations of the data or information available, as risks to public health or welfare may involve the frontiers of scientific or medical knowledge. At the same time, the Administrator mu st exercise reasoned decision making, and avoid speculative inquiries.” That is what the Administrator has done, and the result is neither a political nor a scientific decision, but an exercise of judgment that is informed by and strongly grounded in the science and the law.

Comment (1-34) : A number of commenters (0152, 0718, 0726, 0730, 2895, 3502.1ANPRref1, 4036.5, 5844, 6708, and 9418) state EPA can and should act in the face of un certainty, as the Agency has done in the past, to prevent threat to human health and the environment consonant with the bedrock foundation of the CAA, a law that is anchored in protective action. Some of these commenters also indicate it is wise for EPA to act in light of the number of scientists whose findings support EPA’s actions.

Response (1-34) : As discussed in Section II of the Findings, the Admi nistrator is required to protect public health and welfare, but she is not asked to wait until harm has oc curred. EPA must be ready to take regulatory action to prevent harm before it occurs. Section 202(a)(1) requires the Administrator to “anticipate” “danger” to public health or welfare. The Administrator is thus to consider both current and future risks. See above comment response regarding the Administrator consideration of uncertainty.

Comment (1-35) : A commenter (2895) recommends that EPA make polic y based on probability rather than uncertainty. The commenter cautions that making policy based on uncertainty would require EPA to consider all future possibilities irrespective of their likelihood. It suggests EPA rely on the most likely scenarios in its decision-making.

Response (1-35) : The Administrator made her findings based on the totality of scientific evidence, some of which was assessed as being virtually certain or had been assi gned high or very high probabilities in the assessment literature, while other evidence was less certain. The Ad ministrator considered all of this evidence, and did not restrict herself to only the evidence of hi ghest likelihood. She weighted and evaluated those scenarios, as well as scenarios of lesser probability, to evaluate the full range of risks and potential harms that climate change could lead to in the United States. See Section IV.B. of the Findings for her rationale in finding endangerment of both public health and welfare.



### 34Comment (1-36) : Several comments (2057, 3291.1, 3906) argue IPCC’s le xicon for communicating uncertainty is flawed. One comment argues that the probab ility assessments of IPCC reflect the judgment of the authors rather than empirical data, and that the IPCC probability assessments express greater certainty than the actual method supports. Specifically, it notes when presenting statistical information, IPCC uses a 90 percent confidence interval, not the 95 percent confidence interv al that is standard scientific practice, making the results look more robust than they actually are.

Response (1-36) : IPCC’s application of uncertainty lexicon is prim arily based on the results of probabilistic assessments and some are based on an elicitation of expert views rather than quantitative analysis. However, given the level of expertise of IPCC-nominated lead authors as well as the multi-stage review process, we find that the expert elicitation conducted by IPCC is legitimate. More information regarding the approach to addressing uncertainties in IPCC (2007) is available in the document Guidance Notes for Lead Authors of the IPCC Fourth Assessment Report on Addressing Uncertainties , which is located in Appendix A of this volume

The assertion that IPCC strayed from scientific conven tion in its use of 90% confidence intervals is baseless. In statistical analysis, 5 th and 95 th percentiles are commonly used because they refer to a result two standard deviations from the mean in a normal di stribution – in other words, they help show how much variation there is from the average or mean . The probabilistic assessments performed by IPCC are not statistical analyses but rather reflect expert elic itation and/or the results of quantitative analysis. These are two different processes. There is no rationale for the IPCC to adopt the statistical convention for describing a normal distribution to characterize uncertainty.

Comment (1-37) : A number of commenters (e.g., 0659, 1312, and 10855) not e that scientists were offering warnings about global cooling in the 1970s (some cite examples), and therefore question the credibility of climate change science and the plausibility of projections.

Response (1-37) : Climate change statements and predictions made in the 1970s were not based on the current state of climate change science knowledge. Since the 1970s, ma rked improvements have been made in scientific understanding of climate change due to important theoretical disc overies, new and improved observing systems, and vastly superior computational a nd modeling resources. Furthermore, a recent study (Peterson et al., 2008) surveyed scientific papers published between 1965 and 1 979 that projected future climate trends, and found that only 12% inclined towa rd future cooling, while most papers predicted future warming. The study concluded: “There was no scientific consensus in the 1970s that the Earth was headed into an imminent ice age. Indeed, the possibility of anthropogenic warming dominated the peerreviewed literature even then.”

Comment (1-38) : Many commenters (e.g., 0223, 0261, 0524, 0718, 0726, 1318.1, 3361.1, 3454.1, 3455.1, 3606.1, 3893, 3940, 4136, 4171, 4247, 4841, 6055, 6227, 6465, 6525, 6599, 6622, 6936, 6937, 9298, 9520, 9642, 10003, 10033, 11198, 11249, 11305) argue that the state of the science (i.e., breadth and depth) is strong enough to defend the Proposed Findings and that th ere exists “incontrovertible evidence...that carbon dioxide and other global warming pollution endangers public health and welfare.”



### 35Response (1-38) : We agree with the commenters that the state of the science is strong enough to support the finding.

Comment (1-39) : Commenters (e.g. 3080, 10160) argue EPA should consid er the option of waiting some period of time to make an endangerment determination because the sci entific uncertainty is too great to make a determination at present.

Response (1-39) : The Administrator has determined the scientific in formation available now is sufficiently robust for making an endangerment finding. For more informati on on the Administrator’s rationale for making her finding, please refer to Section IV.B. of the Findings.

Comment (1-40) : Several commenters (3283.1, 4173ANPRref1) specifically reference the 2001 NRC report Climate Change Science: An Analysis of Some Key Questions (NRC, 2001a) to assert that the science is too uncertain for the Administrator to make a reasone d judgment on endangerment. They note that report stated, for example, that a causal linkage between the buildup of greenhouse gases in the atmosphere and the observed climate change during the 20 th century cannot be unequivocally established (owing to the still uncertain natural variability inherent in the clim ate record and the uncertainties in the time histories of the various forcing agents and particularly aeros ols) and that the health consequences of climate change are poorly understood. One commenter argues that since the 2001 NRC report, areas of concern noted by NRC like the importance of aerosols ha ve become less well understood. The commenter concludes that EPA has not demonstrated the scien tific information supporti ng the endangerment finding is adequate.

Response (1-40) : We disagree that the scientific information supp orting the endangerment finding is inadequate. Please refer to Section III.A. of the Findings for a discussi on of the science on which the decisions are based and Section IV.B. for a discussion of how the science w as weighed and evaluated, including uncertainty.

The 2001 NRC report is one of the assessment reports we relied upon in the TSD. The findings of this report still represent the state of the science. However, this report is now over eight years old and many of its findings have been updated in more r ecent assessment literature. For example, though NRC (2001a) refers to “still uncertain natural variability,” th e IPCC (Hegerl et al., 2007) states that it is very unlikely (less than 10% proba bility) that the global pattern of warming observed during the past halfcentury is due to only known natural external causes (solar activity and volcanoes) and that globalmean unforced variations on multidecadal timescales are likely (66–90% probability) to be smaller than the 20 th century global-mean increase in surface temperat ure. Regarding the “uncertainties in the time histories of the various forcing agents and particularly aerosols” indicated by NRC (2001a), we note that IPCC (2007d) states with very high confidence (a t least 90% certainty) that the global average net effect of human activities since 1750 has been one of wa rming, with a radiative forcing of +1.6 (+0.6 to +2.4) W/m 2 . Though uncertainties in aerosol contributions to radiative forcing remain, the commenter’s assertion that aerosols have become less understood since 2001 is incorrect. In fact, IPCC (Forster et al., 2007) states, “Atmospheric models have impr oved and many now represent all aerosol components of significance. Improved in situ , satellite and surface-based measurements have enabled verification of global aerosol models.” Although NRC (2001a) stated th at health effects of climate change are “poorly



### 36understood,” we note that many health effects of climate change are well-documented in the most recent assessment literature (Confalonieri et al., 2007; CCSP, 2008b; Karl et al., 2009), as described in Sections 7 and 8 of the TSD.

Finally, although NRC (2001a) contains the above cau tionary statements which have been updated in more recent assessment reports, we note that the re port begins with the following statement which continues to be broadly consistent with th e current assessment literature: “Greenhouse gases are accumulating in Earth’s atmosphere as a result of human activities, causing surface air temperatures and subsurface ocean temperatures to rise. Temperatures are, in fact, rising.”

Section II of the Findings discusses how EPA consider s uncertainty in making an endangerment decision. The apparent criteria suggested by the commenter are not appropriate for the reasons discussed in Section
II.

Comment (1-41) : At least one commenter (3347.1) notes that many of EPA’s predictions depend on a causal chain of multiple events occurring. The comment explains that th e likelihood of a series of uncertain events is the product of their probabilities, which approaches zero as the number of events in the series increases. It concludes that EPA cannot base a reasoned decision, ev en one that does not require certainty or actual proof, on data, studies, and assumptions th at are of little or unknown reliability.

Response (1-41) : The commenter is referring to the concept of prop agation of uncertainty, which does not change the legitimacy of scientific projections described in the TSD or considered in the Administrator’s endangerment finding. The document Guidance Notes for Lead Authors of the IPCC Fourth Assessment Report on Addressing Uncertainties (see Appendix A of this Volume) directs IPCC (2007) authors to the Third Assessment uncertainty guidance document (M oss and Schneider, 2000) and an IPCC workshop report (Manning et al., 2004) which discuss this issue. IPCC authors are advised to take into account propagation of uncertainty and other potential sources of uncertainty in their statements and findings. In other words, the fact that uncertainty accumula tes through a chain of causation is reflected and communicated in the results presented in the assessment literature (e.g., by describing ranges, levels of confidence/certainty).

The IPCC workshop report (Manning et al., 2004) states that “There is a recognized cascade of uncertainty when proceeding through projected climat e change, projected effects of that change, and projected adaptation or mitigation responses. See Fi gure 3-1. Dealing with this uncertainty cascade is complex and it should not be assumed that uncertainties add in the sense of being statistically uncorrelated.” Figure 3-1 shows five uncertainties between emissions and impacts: uncertainty in emissions scenarios, in carbon cycle response, in climat e sensitivity, in regional scenarios, and in range of possible impacts. This figure demonstrates two key issues. One is that there are a limited number of uncertainties in the chain, in contrast to the implication of the commenter that the number of events in this series is large. Most of the uncertainties in clim ate change do not depend on each other: for example, uncertainty in the dependence of Atlantic hurri cane frequency on sea surface temperature changes is mostly independent from the uncertainty in the relati on between extreme heat events in the Midwest and global average temperature increase. Second, the uncer tainty is not a “does it exist/does it not exist” causal chain of events as described by the commenter, but rather a set of uncertainties that are each distributions of outcomes around a central value, w ith dependence on the previous uncertainty in the chain. This means that the central estimate (or median) outcome has little dependence on the number of uncertainties in the chain, and also that both tails of these distributions grow as the number of uncertainties increase. The final “range of impacts” s hows that inclusion of uncertainty should lead to



### 37higher estimates as often as lower estimates. Figure 10.26 of Meehl et al. (2007) demonstrates these attributes of combining uncertainties: it shows globa l mean temperature projections from six different emission scenarios, including uncertainty in carbon cycle response and in the structural parameters of the models. There are larger uncertainties when including carbon cycle uncertainties on top of the model uncertainties, but these uncertainties lead to higher temperatures than the original distribution as well as lower temperatures.

The CCSP (2009) report Best Practice Approaches for Characterizing, Communicating, and Incorporating Scientific Uncertainty in Decisionmaking suggests that propagation of uncertainty can be constrained by running an analysis over and over agai n on a fast computer, using different input values, from which it is possible to compile the results into probability distributions. This approach is termed “stochastic simulation” or “Monte Carlo.” A number of the studies assessed by the IPCC and CCSP reports employed such an approach to quantitativ ely combine the cascade of uncertainties into output probability distributions.

Therefore, we conclude that the assessment litera ture has appropriately addressed the well-recognized “cascade of uncertainty,” and that the underlying sci ence is sufficiently well characterized for use in making reasoned decisions.

Comment (1-42) : At least one commenter (3722) argues the Administrato r’s conclusions in the Proposed Findings overstate scientific understanding of climate change. The co mment specifically refers to the following two

statements:

It is clear that current and projected leve ls of greenhouse gases and resultant climate change are already and adversely affecting, and will continue to adversely affect, public welfare within the meaning of the Act.

The evidence points ineluctably to the conclu sion that climate change is upon us as a result of greenhouse gas emissions, that climatic changes are already occurring that harm our health and welfare, and that the effect s will only worsen over time in the absence of regulatory action.

The commenter concludes that these unequivocal warn ings mischaracterize the state of climate science and obscure the fundamental uncertainties.

Response (1-42) : The specific language highlighted by the commenter has been revised in the Findings. The key science from the assessment literature underlying these statements remains strong and accurate.

Comment (1-43) : Many commenters (e.g. 0525, 1312, 2895, 2923, 3450.1, 3507.1, 3553.1, 3569.1, 3722, 3915, 4092, 4395,7019, 11225, and 11290) question the notion of scientific consensus around the conclusion of humaninduced global warming and/or mention the existence of large numbers of dissenting scientists who have reached alternative findings to those of IPCC and CCSP/USGCRP, indicating profound scientific uncertainty. As evidence of these dissenting viewpoints, commenters refer to:

T he 35+ scientists who participated in the NIPCC.



### 38T he U.S. Senate Minority Report U pdate Report of March 17, 2009 (see http://epw.senate.gov/public/index.cfm?FuseAc tion=Minority.Blogs&ContentRecord_id=2674e6 4f-802a-23ad-490b-bd9faf4dcdb7 ), which states that more than 700 prominent international scientists, including many current and former me mbers of the IPCC, disagree with the man-made global warming claim.
A group of more than 31,000 scientists (including 9,021 Ph.D.s) who signed a petition from the Oregon Institute of Science and Medicine ( http://www.oism.org/pproject/ ) urging the U.S. government to reject the global warming agreement that was written in Kyoto, Japan, in December, 1997, and any other simila r proposals (petition began in 1998).
A n open letter to the Secretary-General of th e United Nations in 2001, signed by over 100 scientists, statisticians, and econom ists, urging the United Nations to direct its efforts toward dealing with climate change impacts and adaptati on, rather than climate change prevention.
T he Manhattan Declaration (ISCC, 2008: http://www.climatescienceinternational.org/i ndex.php?option=com

_ cont ent&task =view&id=37& Itemid=1 ), dated March 4, 2008, in which scientists , researchers, economists, and policymakers asserted the scientific consensus on climate cha nge is overstated. The signatories recommended that world leaders reject the IPCC views and declare that “human-caused climate change is not a global crisis.” The Declaration w as signed by over 600 individuals.

At least one commenter (3283.1) indicates that EPA cites consensus without specifically describing the meaning and context of that consensus.

Response (1-43) : A response to the NIPCC report cited by commenter s as evidence of dissenting scientific viewpoints can be found at comment response (1-12) in Section 1 of this volume. In addition, please refer to Section III.B. of the Findings for a discussion of the science on which the decisions are based.

We first note that the strength of the science is not dete rmined by petitions or lists of names; rather it is determined by the detailed examination of the fully bo dy of literature, as has been done in the assessment reports, and as has been done in response to comme nts throughout the volumes of this Response to Comments document. Such a review demonstrates that the scientific basis for the TSD and the Administrator’s determination is sound. We respond to the specific petitions below:

Regarding the U.S. Senate Minority Report listing dissenting scientists, we note that the Center for Inquiry 1 recently released a study (Center for Inquiry, 2009: http://www.centerforinquiry.net/opp /news/senat e_minority_report_on_global_warming_not_credible/ ) which found that 80 percent of “dissenting scientists ” in said report had not published peer-reviewed climate research. The study also noted that of the report’s purported list of 700 scientists:

S lightly fewer than 10 percent coul d be identified as climate scientists.
A pproximately 15 percent had published in the rec ognizable refereed literature on subjects related to climate science.
A pproximately 4 percent appeared to favor the current IPCC-2007 consensus and should not have been on the list.

The Center for Inquiry draws the follo wing conclusions based on its analysis:

1 The Center for Inquiry ( http://www.centerforinquiry.net/opp/about ) describes itself as a nonpartisan, nonprofit 501(c)(3) organization that encourages evidence-based inquiry into science, pseudoscience, medicine and health, religion, ethics, secularism, and society

.



### 391) We think it is highly unlikely that a gr owing fraction of top climate scientists are becoming increasingly skeptical of human causation of global warming.

2) We think that the title “Senate Minority Report” is technically appropriate, but grossly understated. That report’s list does contain the names of some outstanding scientists, including at least one distinguished meteorologist. However, when weighted against the much larger number of equally outstanding c limate scientists, there is no doubt where the great majority of experts in this field stand.

Though the declarations, petitions, and letters referred to by commenter s demonstrate the existence of dissenting viewpoints, they do not represent the viewpoi nts of the overwhelming majority of the active climate science research community nor provide legi timate scientific evidence to substantiate the alternative points of view presented. For example, though the Oregon Petition is accompanied by a study (Robinson et al., 2007) refuting many conclusions of the climate science assessment literature, it is published in a medical journal unrelated to climate science.

We further note a recent polling study (Doran and Zimmerman, 2009) in which researchers found 75 out of 77 climatologists who “listed climate science as thei r area of expertise and who also have published more than 50% of their recent peer-reviewed papers on the subject of climate change” believe that human activity is a significant factor in changing me an global temperatures. The study concludes: “ It seems that the debate on the authenticity of global warming and the role played by human activity is largely nonexistent among those who understand the nuances and scientific basis of long-term climate processes.”

It is also relevant to point out that many majo r national and international scientific societies and academies have endorsed or expressed support for the findings and conclusions of the assessment literature cited by EPA in official stat ements. The following are some examples:

S ince 2005, the Academies of Science for the G8+5 countries have issued joint statements on the threat of climate change.

o The 2005 statement reported: “...there is now strong evidence that significant global warming is occurring. The evidence comes fro m direct measurements of rising surface air temperatures and subsurface ocean temperatures and from phenomena such as increases in average global sea levels, retreating glaciers, and changes to many physical and biological systems. It is likely that most of the warming in recent decades can be attributed to human activities (IPCC 2001). This warming has already led to changes in the Earth's climate.” http://www.nationalacadem ies.or g/onpi /06072005.pdf

o The 2007 statement finds: “Recent research st rongly reinforces our previous conclusions. It is unequivocal that the climate is cha nging, and it is very likely that this is predominantly caused by the increasing human interference with the atmosphere. These changes will transform the environmental conditions on Earth unless counter-measures are taken.” http://www.nationalacademies.org/includes/G8Statement_Energy_07_May.pdf

o The 2008 statement notes: “In 2007 the Inte rgovernmental Panel on Climate Change (IPCC) reaffirmed that climate change is happening and that anthropogenic warming is influencing many physical and biological systems” http://www.nationalacademies.org/includes/climatechangestatement.pdf

o The 2009 Statement reports: “The IPCC 2007 Fourth Assessment of climate change science concluded that large reductions in th e emissions of greenhouse gases, principally carbon dioxide (CO 2 ), are needed soon to slow the increase of atmospheric



### 40concentrations, and avoid reaching unacceptabl e levels. However, climate change is happening even faster than previously estimated; global CO 2 emissions since 2000 have been higher than even the highest predicti ons, Arctic sea ice has been melting at rates much faster than predicted, and the rise in the sea level has become more rapid. Feedbacks in the climate system might lead to much more rapid climate changes.” http://www.nationalacademies.org/includes/G8+5energy-climate09.pdf
T he American Meteorological Society in 2007 stated: “...there is adequate evidence from observations and interpretations of climate simulations to conclude that the atmosphere, ocean, and land surface are warming; that humans have signi ficantly contributed to this change; and that further climate change will continue to ha ve important impacts on human societies, on economies, on ecosystems, and on wildlife thr ough the 21st century and beyond.” We further note the AMS mentions: “This statement is consiste nt with the vast weight of current scientific understanding as expressed in assessments and re ports from the Intergovernmental Panel on Climate Change, the U. S. National Academy of Sciences, and the U. S. Climate Change Science Program.” http://www.ametsoc.org/policy/2007climatechange.html
T he American Chemical Society writes: “There is very little room for doubt that observed climate trends are due to human activities. The threats are serious and action is urgently needed to mitigate the risks of climate change.” We note that ACS cites IPCC (2007) as a core reference for its statement which is valid for the period 2007–2010. http://portal.acs.org/portal/acs/corg/content?_nfpb=true&_pa geLabel=PP_SUPERARTICLE & no de_id=1907&use_sec=false&sec_url_var= region1&__uuid=34a7a d19-6b0b-4720-a706- e25e4e609d6d
T he American Geophysical Union in 2007 writes: “The Earth's climate is now clearly out of balance and is warming...With climate change, as with ozone depletion, the human footprint on Earth is apparent. The cause of disruptive climat e change, unlike ozone depletion, is tied to energy use and runs through modern society. Solutions will necessarily involve all aspects of society.” http://www.agu.org/outreach/science_po licy/positions/cli ma te_change2008.shtm l

.

We further note that Alan Leshner, Chief Executiv e Officer of AAAS, submitted a comment (in response to EPA’s ANPR, 1009.1), in which he states:

...AAAS seeks to offer requested comments on “the best available science for purposes of the endangerment discussion.”

The AAAS Board of Directors approved a statement on climate change in December 2006 based largely on the Third Assessment Report of the Intergovernmental Panel on Climate Change (IPCC) and the 2005 Joint National Academies’ statement. The statement, in part, recognizes:

“The scientific evidence is clear: global climate change caused by human activities is occurring now, and it is a growing threat to society. Accumulating data from across the globe reveal a wide array of effects: rapidly melting glaciers, destabilization of major ice sheets, increases in extreme weather, rising sea level, shifts in species ranges, and more.”

Since the statement was released, the IPCC F ourth Assessment Report has been released, finding greater observed and projected climate change impacts, many of which are cited in the ANPR. These are also consistent with the AAAS statement:

“As expected, intensification of droughts, heat waves, floods, wildfires, and severe storms is occurring, with a mounting toll on vulnerable ecosystems and societies. These



### 41events are early warning signs of even more devastating damage to come, some of which will be irreversible.”

These and other impacts identified in the ANPR reflect the current state of the science, as articulated in the consensus documents of th e IPCC. This science shows clear impacts to society that are occurring now and will continue to grow.

We hope that the Environmental Protection Agency continues to rely upon this sound scientific basis as it moves forward on decisions related to climate policy.

In addition, on October 21, 2009, 18 U.S. scientific societies and/or research institutions (including all of those mentioned above) reaffirmed the c onsensus on climate change in a letter 2 to U.S. senators, stating:

Observations throughout the world make it clear that climate change is occurring, and rigorous scientific research demonstrates that the greenhouse gases emitted by human activities are the primary driver. These conc lusions are based on multiple independent lines of evidence, and contrary assertions are in consistent with an objective assessment of the vast body of peer-reviewed science. Mo reover, there is strong evidence that ongoing climate change will have broad impacts on society, including the global economy and on the environment. For the United States, climate change impacts include sea level rise for coastal states, greater threats of extreme w eather events, and increased risk of regional water scarcity, urban heat waves, western wildfires, and the disturbance of biological systems throughout the country. The severity of climate change impacts is expected to increase substantially in the coming decades.

If we are to avoid the most severe impacts of climate change, emissions of greenhouse gases must be dramatically reduced.

These societies and organizations noted that their conclusions “reflect the scientific consensus represented by, for example, the Intergovernmental Panel on Climate Change and U.S. Global Change Research Program.” They also noted that many scientific societies have endorsed these findings in their own statements, including the American Association for the Advancement of Science, American Chemical Society, American Geophysical Union, and American Meteorological Society (as documented above), as well as the American Statistical Association and Crop Science Society of America.

Regarding the claim that EPA cites consensus without describing its meaning and context, we disagree. Our references to consensus in the TSD and the Findings are consistently within the context of describing conclusions of the assessment literatu re upon which we rely, which are, in fact, consensus-based. The Web site ( http://www.globalchange.gov/whats-new ) for the recent USGCRP assessment (Karl et al., 2009) notes it represents “a consensus of 13 agencies de veloped over a year and half.” In its report “Understanding and Responding to C limate Change,” the National Academy of Sciences notes that in its NRC reports on climate change (which are cited in the TSD) they “have assessed consensus findings on the science, identified new avenues of inquiry and critical needs in the observing and computational research infrastructure, and explored opportunities to use scientific knowledge to more effectively respond to climate change.” The 2005 Joint Science A cademies’ statement of 2005 states: “We recognize the international scientific consensus of the Inte rgovernmental Panel on Climate Change (IPCC).” For further information on the process involved for devel oping the consensus conclusions of the IPCC, please refer to comment # 1-14.

2 http://www.ametsoc.org/sss/doc uments/climateletter.pdf



### 42Comment (1-44) : To highlight uncertainty in the science, a number of commenters (0482.1, 0591, 0700.1, 3569.1, 3906,) reference the scientific testimony of the Right H onorable Christopher Walter Monckton, Third Viscount Monckton of Brenchley, before the Energy and Commerce Committee of the House of Representatives on March 25, 2009, and/or follow- on letter (addressed to Congressman Joe Barton and Congressman Edward Markey), which question the links between hum an activities and global warming (Monckton, 2009a, 2009b). One commenter (3906) requests EPA counter any flaws in the letter.

Response (1-44) : We respond to the relevant assertions in Lord M onckton’s testimony and in his follow-up letter to Representatives Markey and Barton in the appropriate Response to Comment volumes. In his letter, Lord Monckton enumerates 50 “red flags” which represent sci entific irregularities in statements or conclusions made by prominent climate scientists, in the assessment literature, and/or in our TSD. We address most of these, but note that some of the “red flags” apply spec ifically to statements made by individuals (e.g., Vice President Al Gore, scientists who testified before Congress, congressional members), which only they could reasonably respond to and which we found to be extraneous to the assessment literature and/or statements in the TSD.

Comment (1-45) : Two commenters (0152, 0718) voice their support for EPA’s Proposed Findings and state that even if the full risks and implications of global warming cannot be quantified, EPA has enough information to take “concrete and serious steps.”

Response (1-45) : We agree with the commenters that the state of the science is strong enough to support the Findings.

Comment (1-46) : Several commenters (2972.1, 3136.1, 3449.1, and 37 47.1) argue that the TSD does not differentiate between the risks to various physical and biological systems specifically resulting from climate change and those due to other non-climate pressures including subsidence, land use, development, and population growth. Some commenters also state that the TSD overemphasizes climate change pressures, when in reality the non-climate pressures are the larger pr oblem. The commenters conclude that because climate change is not the only cause of impacts to physi cal and biological systems and because climate change drivers cannot be measurably isolated from other non-cl imate drivers of impacts, this information cannot be used to support the endangerment finding. Comm enter 3747.1 argues that EPA’s treatment of these baseline assumptions do not meet information quality requirements under the IQA.

Response (1-46) : EPA disagrees with the commenters that the inform ation summarized in the TSD does not differentiate between climate and non-climate drivers of impacts to physical and biological systems, or adequately isolate the observed effects and future risks and impact s attributable to elevated concentrations of GHGs and associated climate change, and this issue is a ddressed extensively throu ghout the TSD and in the Response to Comments document. For example, we d escribe in Section 5(b) of the TSD the importance of considering climate variability and non-climate driv ers (e.g., land-use change, habitat fragmentation) in order to make robust conclusions about the role of anthropogenic climate change in affecting biological and physical systems. The discussions of climate imp acts on the sectors described in Sections 7-15 of the TSD are also summarized with acknowledgement of the non-climate drivers. As an illustration, in



### 43Sections 12 and 14, we summarize findings from th e assessment literature wh ich discuss impacts to coastal areas and ecosystems resulting solely from climat e change or from the interaction of both climate change and non-climate-related stressors. The discussion in the TSD is clear that climate change effects on physical and biological systems do not occur in isola tion, and that climate change, other stressors, and the compounded effects of both will present new threats while also compounding existing stressors.

Further, we disagree with the commenter’s statement that our treatment of this issue is inconsistent with EPA’s Guidelines for Ensuring and Maximizing the Q uality, Objectivity, Utility and Integrity of Information Disseminated by the Environmental Protection Agency . For EPA’s general response to the information quality concerns submitte d during the public comment process, please see Section 5 of this volume. With respect to this specific issue raised by commenter 3747.1 we note that the Guidelines encourage risk assessments to analyze and consider ‘real world situations.’ EPA’s approach of assessing climate change impacts independently but also in the c ontext of other stressors is therefore consistent with the Guidelines . An example of this approach is seen in Section 12(a) of the TSD:

Although climate change is impacting coastal systems, non-climate human impacts have been more damaging over the past century. The majo r non-climate impacts for the United States and other world regions include drainage of coastal wetlands, resource extraction 3 , deforestation, introductions of invasive species, shoreline protec tion, and the discharge of sewage, fertilizers, and contaminants into coastal waters (Nicholls et al., 2007). The cumulative effect of these nonclimate, anthropogenic impacts increases the vulne rability of coastal systems to climate-related

stressors.

EPA also disagrees with the commenters that the TSD inappropriately emphasizes climate change pressures. In Section 5(b) of the TSD, we discuss how the IPCC (Rosenzweig et al., 2007) reviewed a number of studies that linked changes in some physical and biological systems directly to anthropogenic climate change. The IPCC concluded that the observed ch anges in physical and biological systems “likely cannot be explained entirely due to natural va riability or other confounding non-climate factors” (Rosenzweig et al., 2007). Furthermore, future projec tions of climate change and associated risks and impacts in the TSD go even further to isolate the pot ential effects of climate change, where information from the underlying assessment literature often assesses the risks of climate change while holding current conditions constant. We therefore find that the TSD’s treatment of climate and non-climate factors was reasonable and appropriately reflects the conclu sions of the body of scientific literature.

See Response to Comments Volume 9 for our res ponses to comments on how specific climate impacts evidence was considered in the Ad ministrator’s endangerment finding.

Comment (1-47) : A commenter (3747.1) argues that EPA did not adhere to information quality guidelines on reproducibility and transparency because the TSD does not explain how EPA determined the probabilities assigned to climate science and impacts conclusions from the assessment literature. The commenter acknowledges that “adopts IPCC phraseology assessing the likelihood of results” but “never explains how it determined the probability.”

Response (1-47) : Please see Volume 1: Section 5 of this Response to Comments document for EPA’s general response to the information quality concerns submitted durin g the public comment process. Please see previous

3 Resource extraction activities in coastal areas include sand/coral mining, hydrocarbon production, and commercial and recreational fishing.



### 44responses to comments in this section regarding our use of the assessment literature’s uncertainty and likelihood determinations in the TSD, as well as Sec tion 1 of the TSD, which describes EPA’s approach and use of lexicon developed by the IPCC, USGCRP, and CCSP for the consistent communication of uncertainty and likelihood. We note that in climate science, likelihood refers to a probabilistic assessment of some well-defined outcome having occurred or that will occur in the future (IPCC, 2007e). In this sense, likelihood and probability are one and the same . For example, when IPCC authors evaluated the likelihood of certain outcomes and determined then to be “very likely,” the probability of occurrence was 90 to 99%.

Additional information has been added to Box 1.2 in Section 1 of the TSD to more fully describe our approach to the communication of uncertainty and probabilities. Additional detail on these issues is provided in prior responses, particularly with respect to the processes followed by IPCC and other assessment reports to develop the reports, which in cludes determining the uncertainty and likelihood category to assign to the various scientific issues discussed in the TSD.

Our characterization and treatment of un certainty is consistent with EPA’s Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and In tegrity of Information Disseminated by the Environmental Protection Agency . Conveying the same likelihood and probability terminology assigned to climate science findings by the IPCC and USGCRP/ CCSP ensures that the information in the TSD is reproducible (the Guidelines state that “EPA intends to ensure reproducibility according to commonly accepted scientific, financial, or statistical standards”) and transparent (the Guidelines state that “A higher degree of transparency about data and methods will f acilitate the reproducibility of such information by qualified third parties, to an acceptable degree of imprecision”).

Comment (1-48) : Two commenters (3577.1 and 3747.1) discuss how EPA, in general, failed to identify uncertainties in the studies that underlie EPA’s Proposed Finding. Comme nter 3747.1 argues that EPA did not adhere to information quality guidelines by not identifying sources of uncertainty in the climate impacts and models described in the TSD. Specifically, the commenter requests that EPA identify:

T he rationale for the Agency’s d ecision not to identify each of th e uncertainties and ranges in the studies and models that underlie its proposed endangerment finding.
W hat impacts the calculation of recognized un certainties and ranges would have on the outcome of the endangerment finding.

Response (1-48) : Please see Volume 1: Section 5 of this Response to Comments document for EPA’s general response to the information quality concerns submitted during the public comment process. In addition, please see previous responses to comments in this section on EP A’s treatment of uncertainty regarding specific issues in the TSD.

We disagree with the comment that the TSD do es not identify uncertainties and present ranges for statements of observed and projected climate ch ange. The TSD describes th e assessment literature’s conclusions regarding the state of the science, discu sses key uncertainties with re gards to specific climate science issues, and presents observations and proj ections in ranges based upon different emission scenarios and other factors specific to a particular climate science issue. For example, Section 6(b) of the TSD states:

The full suite of SRES scenarios (given belo w) provides a warming range of 3.2°F to 7.2°F (1.8°C to 4.0°C) with an uncertainty ra nge of 2.0°F to 11.5°F (1.1°C to 6.4°C). The



### 45multi-model average warming and associated uncertainty ranges for the 2090–2099 period (relative to 1980–1999) for each scenario, as illustrated in Figure 6.7 are shown in Table 6.1:

Table 6.1: Multi-model Average Warming for the 2090-2099 Period

Scenario Average Global Warming by End of Century Relative to ~1990 Uncertainty Range
B1 3.2°F (1.8°C) 2.0°F to 5.2°F (1.1°C to 2.9°C)
A1T 4.3°F (2.4°C) 2.5°F to 6.8°F (1.4°C to 3.8°C)
B2 4.3°F (2.4°C) 2.5°F to 6.8°F (1.4°C to 3.8°C)
A1B 5.0°F (2.8°C) 3.1°F to 7.9°F (1.7°C to 4.4°C)
A2 6.1°F (3.4°C) 3.6°F to 9.7°F (2.0°C to 5.4°C)
A1FI +7.2°F (+4.0°C) 4.3°F to 11.5°F (2.4°C to 6.4°C)

The wide range of uncertainty in these estimat es reflects the different assumptions about future concentrations of GHGs and aerosols in the various scenarios considered by the IPCC and the differing climate sensitivities of the various climate models used in the simulations (NRC, 2001a; Meehl et al., 2007; Karl et al., 2009).

Given that the TSD conveys the uncertainty inform ation associated with the assessment literature’s conclusions, the commenter’s request to identify each of the uncertain ties and ranges in the studies and models that underlie the Findings is unnecessary. Ou r characterization and treatment of uncertainty is consistent with EPA’s Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by the Environmental Protection Agency , because our approach maximizes objectivity (which the Guidelines define as “whether the disseminated information is being presented in an accurate, clear, complete, and unbiased manner, and as a manner of substance, is accurate, reliable, and unbiased”), utility (defined by the Guidelines as “usefulness of the information to the intended users”), and transparency (which allows for “the reproducibility of such information by qualified third parties, to an acceptable degree of imprecision”).

Finally, since the TSD appropriately describes uncer tainties and presents climate science information in ranges, we reject the commenter’s request that EPA reassess the Endangerment Determination.

1.3 Treatment of Adaptation and Emissions Mitigation

Comment (1-49) : A number of commenters (e.g., 2898.1, 3136.1, 3261.1, 3332.1, 3347.1, 3481.1, 3482.1, 3509.1, 3536.2, 3747.1) state that the Proposal and TSD should ha ve explicitly assessed the potential for adaptation to protect public health and welfare from climate change . These commenters argue that observed trends in measures of U.S. health and welfare during a tim e of increasing anthropogenic GHG emissions provide evidence of the potential for adaptation to counteract any negative impacts of climate change, and that EPA overestimated the negative impacts of future climate change by not considering the role of adaptation. A commenter (3126.1), fo r example, asserts that sensitivity to heat has been declining, air quality improving, crop yields increasing, and life sp ans increasing, and that all this indicates that “weather and climate are becoming less and less a negative factor in our overall health and welfare.” According to the commenter, “Our adaptations ha ve improved our ability to handle the negative consequences of weather while enjoying the positive ones.”



### 46In arguing that EPA’s approach to adaptation was inadequate or inappropriate for purposes of an endangerment analysis, many commenters (e.g., 2898.1, 3136.1, 3261.1, 3332.1, 3347.1, 3394.1, 3482.1, 3509.1, 3536.2, 3596.1, 3702. 1, 3704.2, 3705.1, 3747.1) criticize EPA’s characterization in the Proposal of potential policy or societal responses to climate change as speculative. They argue that EPA assumes humans will remain passive without changing their beha vior or attempting to ameliorate the impacts of climate change, while in reality adaptive measures are all but inevitable.

Two commenters (3347.1 and 3509.1) state: 1) that autonomous adaptation affects the severity of impacts; 2) that anticipatory adaptation affects the lik elihood of impacts; and 3) that both should therefore have been considered in the Proposal. One of these commenters (3509.1) characterizes the decision not to consider adaptive effects as “erroneous and clearly arbitrary.”

Finally, several commenters (e.g., 3332.1, 3536.2, and 3747.1) argue that EPA should explicitly consider not just potential adaptations but also mitigation st rategies, including their potential to prevent harm. These commenters assert that it is unreasonable for EPA to assume that society will not undertake mitigation in future.

Response (1-49) : See Section III.C, “Adaptation and Mitigation,” of the Findings for our response to comments on the treatment of adaptation and mitigation in the Findings.

Comment (1-50) : Two commenters (2898.1 and 3394.1) state that EPA should explicitly consider potential private adaptation and critiqued the following statement from the Proposal: “Just as the Administrator would not consider, for example, the availability of asthma medi cation in determining whether criteria air pollutants endanger public health, so the Administrator will not consider private behavior in the endangerment determination at hand.” A commenter (3394.1) states th at the argument is “not a relevant analogy to adaptation in the context of climate” and that th e availability of asthma medication in determining whether criteria air pollutants endanger public health “is not comparable to evolutionary responses in natural ecosystems or to changes in human behavior th at may prevent welfare or indirect health impacts from ever occurring at all.” The commenter also states that “in a variety of other settings, EPA has promulgated regulations and implem ented policies to encourage adaptiv e responses to what might have otherwise developed into adverse pub lic health and welfare impacts.”

A commenter (2898.1) equates EPA’s decision not to c onsider potential private adaptation to a catch-22, stating that EPA’s reasoning for not explicitly consider ing adaptation implies that if people are anticipated to be worse off, that constitutes endangerment, but if people are reasonably anticipated to be better off, albeit due to adaptation, that also constitutes enda ngerment. According to the commenter, this reasoning guarantees an endangerment finding regardless of the risks people actually face. The commenter additionally questions whether EPA’s reasoning imp lies that we are all endangered, all the time, mentioning that there are very few places on Earth where people can be healthy and thriving without forms of private adaptation to climate such as clothing, shelter, and agriculture.

Response (1-50) : See Section III.C, “Adaptation and Mitigation,” of the Findings for our response to comments on the treatment of adaptation and mitigation in the Findings.

The specific statement from the Proposal critiqued by commenters 2898.1 and 3394.1 is not included in the revised discussion of adaptation included in the finding, where we have provided additional clarification on our treatment of adaptation a nd mitigation (see Section III.C). The discussion of



### 47adaptation and mitigation in Section III.C of the Fi ndings encompasses our treatment of all forms of adaptation.

In any case, the commenter presents a false comparis on, assuming that climate change combined with future adaptation would lead to people being bette r off. Nothing in the ev idence before EPA or the comment would lead to that conclusion. The commenter also mischaracterizes EPA’s position—EPA is not determining whether climate or climate change endangers people, assuming we remove all present social circumstances of housing, infrastructure, and the like. EPA’s position is that while future adaptation or mitigation may well occur, it does not change the issue before EPA—does the air pollution endanger public health or welfare, without considering future pla nned adaptation or mitigation? The reasons for this approach are described in Section III.C. of the Findings.

Finally, when we do not respond to comments on specific adaption or mitigation issues in the various volumes of this Response to Comments document, it s hould not be interpreted as our agreeing with the assertions of the commenter. We are not responding because it is not relevant to the Findings; we are not agreeing or disagreeing with the commenter on those issues.

Comment (1-51) : Several commenters (3347.1, 3379.1, 3482.1, 3509.1, 3702.1, 3705.1) cite the following quotation from the TSD (itself a quotation from the CCSP report on hu man health): “The United States is certainly capable of adapting to the collective impacts of climate change.” These commenters argue that the quotation is highly relevant to the likelihood and severity of public health– or welfare-related effects of climate change in the United States, that an endangerment analysis must seek to determine what type and extent of harm is likely to occur, and that this question cannot be answered while ignoring the role of adaptation.

Response (1-51) : See Section III.C, “Adaptation and Mitigation,” of the Findings for our response to comments on the treatment of adaptation and mitigation in the Findings.

With respect to this specific quote, it is incorrect to interpret its inclusion in the TSD as evidence that there are not risks to health and welfare. Furthe r, the commenters only incl uded one sentence of the section from which the quotation was drawn. The follo wing sentence in the TSD states: “However, there will still be certain individuals and locations where the adaptive capacity is less and these individuals and their communities will be disproportionally impacted by climate change.” For EPA’s responses to comments and literature provided on specific climate impacts summarized in the TSD, please refer to the appropriate Response to Comment volumes.

Comment (1-52) : A commenter (3347.1) asserts that it is inconsistent and arbitrary for EPA to consider autonomous adaptation trends by non-human life forms while disregarding human adaptation in reaching its conclusion. The commenter argues that the TSD sel ectively describes some ad aptive responses that are likely to occur as a result of climat e change, such as a shift of species to northern and higher elevations, while omitting potentially more important adaptive res ponse. The commenter states that “EPA’s decision to broadly exclude other adaptive responses, includi ng autonomous and anticipatory adaptive responses that could undermine the proposed endangerment finding, is arbitrary and represents a bias at odds with an objective with an objective analysis of risks.”



### 48Response (1-52) : See Section III.C, “Adaptation and Mitigation,” of the Findings for our response to comments on the treatment of adaptation and mitigation in the Findings.

The TSD summarizes key observations from the assessment literature regarding natural autonomous adaptation; however, the decision not to include information on planned adaptation in the TSD was not based on arbitrary or biased reasons. Autonomous adaptation 4 is distinct in several ways from potential planned adaptations and merits different treatment.

First, observed non-human autonomous adaptations are part of the physical changes among ecosystems and wildlife in response to climate change. Second, in contrast to planned adaptations, autonomous adaptations are not a conscious response by society to climatic stimuli. Rather, they are natural physical responses to changes in the climat e that occur independently of an y potential human responses, thus representing an important strand of physical evidence that global climate is changing.

There are cases in the TSD where some degree of human autonomous adaptation is accounted for. These cases occur where the literature on which the TSD relies already u ses assumptions about autonomous adaptation when projecting the future effects of cl imate change. Such cases are noted in the document.

Comment (1-53) : A commenter (3702.1) asserts that other agencies routin ely consider adaptive behavior in their regulatory decision-making and stated that NOAA recently considered the past and likely future responsible behavior of ribbon seals in deciding whether to lis t the seal species as threatened or endangered.

Response (1-53) : See Section III.C, “Adaptation and Mitigation,” of the Findings for our response to comments on the treatment of adaptation and mitigation in the Findings. In addition, see the previous response to comments regarding autonomous adaptation.

We further note that the legal requirements governing an endangerment analysis under Section 202(a) of the CAA are distinct from those governing different types of actions undertaken by other agencies. Section II of the Findings discusses the criteria under Section 202(a), and Section III.C. discusses adaptation and mitigation.

Comment (1-54) : A commenter (3136.1) argues that the TSD’s “use of [ hurricane] Katrina as an ‘adaptation is inadequate’ example completely misses the point.” The comme nter states that Katrina “was in no way a climatologically remarkable storm in New Orleans” and was “more an indictment of government emergency response than our adaptive capacity.”

Response (1-54) : See Section III.C, “Adaptation and Mitigation,” of the Findings for our response to comments on the treatment of adaptation and mitigation in the Findings.

4 IPCC defines autonomous adaptation as adaptation that does not constitute a conscious response to climatic stimuli but is triggered by ecological changes in natural systems and by market or welfare chan ges in human systems. Also referred to as spontaneous adaptation.



### 49We note that the question of whether hurricane Katrin a was “climatologically remarkable” is not germane to our reasons for determining that planned adap tation and mitigation are outside the scope of the endangerment finding. For discussion of hurricane Katr ina in relation to climate change and climate change impacts, please refer to Sections 7(b), 12(a ), 12(b), and 13(b) of the final TSD. For EPA’s responses to comments on extreme weather events in relation to climate change, please refer to Volume 2, Section 2.6: “Extreme Weather Events,” and Volume 4, Section 4.5, “Projections of Extreme Weather Events.”

Comment (1-55) : Two commenters (2818 and 4036.5) state that adaptation is not sufficient to prevent harm from climate change. A commenter (2818) argues that adaptation is important but inadequate on its own. The commenter states that though they think adaptation is important, “all the adaptation in the world is not going to save us, unless we also work to get the c limate stabilized.” Another commenter (4036.5) states that there are some impacts of climate change for wh ich adaptation is simply not an option and cited as an example injuries related to extreme weather events that are simply not treatable with medication.

Response (1-55) : See Section III.C, “Adaptation and Mitigation,” of the Findings for our response to comments on the treatment of adaptation and mitigation in the Findings.

Comment (1-56) : A commenter (3747.1) argues that EPA ignored the potential impacts of adaptive behaviors or GHG mitigative actions in reducing impacts to humans a nd the environment, therefore not adhering to information quality guidelines. The commenter argues th at EPA’s lack of consideration on adaptation and mitigation resulted in vulnerabilities being overstated and conclusions th at were biased and incomplete.

Response (1-56) : See Section III.C, “Adaptation and Mitigation,” of the Findings and previous responses to comments in this section regarding EPA’s treatment of adaptation and mitigation in the Findings. EPA has provided a high level of transparency and c onsistency in accordance with EPA’s Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and In tegrity of Information Disseminated by the Environmental Protection Agency. Through the clear description of our treatment of adaptation and mitigation in the Introduction of the TSD, we have been transparent and open with the approach used. This approach is also consistent with the Guidelines because the TSD was peer-reviewed by external climate science experts. Given our approach to the sci entific literature (described in Section III.A of the Finding), the purpose of the federal expert review wa s to ensure that the TSD accurately summarized the conclusions and associated uncertainties from the asse ssment reports. Their unbiased and comprehensive review of the TSD revealed that our treatment of adaptation and mitigation was appropriate and sound, and that we appropriately applied and summarized the information from the assessment literature. This ensured that the climate change risk and impact s information was appropriately presented and that vulnerabilities were not overstated.

Comment (1-57) : Commenter 2898.1 cites a presentation by Indur Goklany (20 06) to argue that mortality in the U.S. due to extreme weather events is currently below its peak le vels of a few decades ago, and that a reversal of these trends is not reasonably anticipated, even if one assumes that global warming will affect extreme weather. The commenter agrees with EPA that heat waves are likely to become more intense but asserts



### 50that more heat waves will not lead to more heat-wave related mortality, due to adaptive actions. The commenter cites a paper (Davis et al., 2003) which they claim shows that as urban air temperatures have increased, due to expanding urban heat islands, cities where hot weather is common have had practically no heat-related mortality.

Response (1-57) : See Section III.C, “Adaptation and Mitigation,” of the Findings for our response to comments on the treatment of adaptation and mitigation in the Findings.

See Volume 5 of the Response to Comments document for EPA’s response to the specific comments and literature provided on heat mortality and morbidity.

1.4 Consideration of Net (Adverse and Beneficial) Effects

Comment (1-58) : A number of commenters (e.g., 3384.1, 3411.1, 3440.1, 3449.1, 3473.1, 3481.1, 3530.1, 4395, 7020, 9636, 10144, 11315, 11449) argue that EPA failed to adequately identify and consider the potential benefits of elevated atmospheric GHG concentrations and climate change. One commenter (11315), for example, states: “EPA acknowledges that there are beneficial impacts from increase [sic] carbon dioxide (the dominant man induced greenhouse gas) but fails to adequately identify them or their significance.” Other commenters (3283.1, 3324.1, an d 3340) contend that the TSD contai ns biased language that focuses on the worst-case potential harm, over-emphasizes negative impacts, and has not adequately attempted to evaluate the potential benefits of climate change.

A number of commenters (e.g., 0400, 0401.3, 0455, 1216.1, 1561.1, 2157, 2750, 2883.1, 2893, 3071, 3214.1, 3261.1, 3261.2, 3307.1, 3395.1, 3722, A29, 8978.1, 10158, 1217.1, 11315, 11492) state that the benefits of elevated atmospheric GHG concentrations and climate change outweigh the adverse effects. Some of these commenters assert that climate change will result in net economic benefits for the United States. In addition, commenters describe a variety of sector-specific benefits that they believe will result from elevated atmospheric CO 2 concentrations and warming temper atures, including: reduced health impacts from cold temperatures that, according to commenters, could outweigh increased risks from heat (e.g., 0400, 0798, 3307.1, 3384.1, 3411.1, 4632R51, 9946, 11453.1); a potential for fewer tropical cyclones (e.g., 0400); positive effects on plant grow th (e.g., 0455, 2883.1, 3 061, 3214.1,); increased crop and food production (e.g., 0798, 1216.1, 1217.1, 1561.1, 3214.1, 3261.1, 3395.1, 4632R51, 10158, 9061.1); and decreases in heating u se and costs (e.g., 2883.1, 4632R51) . Several commenters (e.g., 3224) states that warming to date has been accompanie d by increasing life spans, increases in GDP, and increases in food production/supplies despite increas es in population. Anot her commenter (0700.1) asserts that temperature differentials that cause storms diminish with warming.

Response (1-58) : We reviewed the TSD in light of this comment, a nd find that the TSD summarizes both beneficial and adverse impacts, drawing from the findings in th e assessment literature. The TSD includes information regarding potential benefits of warm ing with respect to heat- and cold-related mortality in Section 7; effects of elevated CO 2 concentrations on the agriculture and forestry sectors in Sections 9 and 10; and impacts of warmer temperatures for energy use, incl uding heating requirements, in Section 13. Thus, EPA does not agree that the TSD contains biased langua ge or does not provide a balanced and reasonable evaluation of both the beneficial and adverse impacts of climate change.



### 51The impacts of changes in tropical cyclone activity are discussed in Section 7(b) of the TSD. Potential changes in storms are addressed in Section 6(e) of the TSD, and see also our responses to comment in Volume 4 of this Response to Comments document.

We received many comments and, in some cases, additi onal literature related to the characterization in the TSD of specific impacts associated with elevated atmospheric GHG concentrations. For EPA’s responses to these comments, please refer to the appropriate Response to Comment volumes. For EPA’s responses to comments pertaining to the method used by the Administrator to find endangerment, please refer to Volume 9.

Comment (1-59) : Several commenters (e.g., 0603.1, 3411.1, 3607.1, 3722, 10144) state that EPA did not explicitly enough quantify positive and negative effects from elevat ed GHG concentrations and climate change to adequately support the Proposed Findings. A commen ter (3603.1), for example, states that EPA should more thoroughly characterize, using numbers and ra nges, the health and welfare impacts. Another commenter (3722) states that EPA should have evaluated in detail, in a balanced and complete fashion, the net effect of warming temperatures in determining whether warming will result in adverse health effects, taking into account both the positive and negative impacts. Two commenters (3322.2, 10144) request that EPA assess and quantify the beneficial impact on climate change and the resultant improvements to public health/safety and welfare associated with regulating GHG emissions under the CAA.

Response (1-59) : We disagree that we did not explicitly enough quantif y the effects of elevated GHG concentrations and associated climate change, including development of numbers and ranges. Our treatment of positive and negative effects from warming and other climatic chang es is consistent with the assessment literature and fully appropriate for purposes of an endangerment finding. We quantified impacts where possible, but we did not do so in cases where it cannot be done in a scientifically rigorous manner. The assessment literature on which the TSD is primarily based do es not numerically quantify the impact of every observed and projected change due to elevated GHG c oncentrations and associated climate change, nor does the TSD.

As demonstrated by the assessment literature, it is possi ble and useful to assign confidence levels to many impact statements, findings, and projections. For this reason, the TSD employs terminology consistent with the assessment reports upon which it is pr imarily based to communicate likelihoods and uncertainties for impacts in each sector (see Section 1.2 of this volume and Box 1.2 of the TSD for a discussion of communication of uncertainty in the IPCC and CCSP/USGCRP reports).

For a discussion of key issues the Administrator cons idered when evaluating the state of the science, including her consideration of the nature and pot ential magnitude of the impacts (both adverse and beneficial) across all climate-sensitive elements of public health and welfare, please refer to Volume 9 of this Response to Comments document and Section IV.B of the Findings. For EPA’s responses to comments on impacts in specific sectors, please refer to the appropriate Response to Comment volumes.

1.5 Information Quality Act Require ments for Independent Assessment

Comment (1-60) :



### 52A number of commenters argue that EPA has not complied with the objectivity, utility, and integrity guidelines of the IQA and EPA Guidelines for Ensuring and Maximizi ng the Quality, Objectivity, Utility and Integrity of Information Disseminated by the EPA. 5 Many commenters went on to explain specific concerns regarding information quality, including EPA’s use of the synthesis literature reports.

Response (1-60) : EPA has fully complied with the requirements of the IQA (also referred to as the Data Quality Act) and the CAA. The IQA requires the Office of Manage ment and Budget and federal agencies to issue guidelines that “ensur[e] and maximize[e] the quality, objectivity, utility, and integrity of information (including statistical information) disseminated by Federal agencies” (Public Law 106-554; 44 U.S.C. 3516, note). The IQA does not impose its own standard of “quality” on agency information; instead, it requires only that an agency “issu e guidelines” ensuring data quality. Fo llowing guidelines issued by the Office of Management and Budget, EPA releas ed its own guidelines to implement the IQA ( http://www.epa.gov/QUALITY/informationguidelin es/docu m ents/EPA_InfoQuality Guidelines.pdf ). These guidelines describe thorough mechanisms under wh ich the Agency may review data quality. In addition to complying with the requirements of the I QA, EPA has acted consistently with the applicable information quality guidelines.

For EPA’s responses to comments on specific informa tion quality concerns raised by the commenters, please see responses in this Section, other sections in this Volume, and relevant Response to Comment volumes.

Comment (1-61) : One commenter (3747.1) argues that EPA has not complied with the IQA and submits 13 separate information quality concerns (labeled “requests for correction” by the commenter) related to the following topics: peer review; treatment of uncertainty and ranges; selection of cited studies; transparency of available data; transparency in observational data; transparency in model limitations; baseline assumption errors; adaptation/mitigation; defining likelihood; misapplication of IPCC storylines; availability of data; precipitation; and sea level rise. For each topic, the commenter provides an argument, along with specific information quality concerns regard ing the data and information used in the TSD and Findings. In their submission, the commenter (3747.1) states: “We recognize that EPA has the right to respond to these findings with its response to comment s as these RFCs [requests for correction] are being submitted within an open comment period. [The comme nter] is willing to forego an individual response and, to the contrary, believes that stakeholders and the public would benef it from EPA’s responses and corrections.”

Response (1-61) : As a preliminary matter, other commenters, such as 3394.1, suggest that the comments were filed as “requests for correction” (RFCs) during this proposal. Any such RFCs in this in stance have been treated as comments on the proposed rule and have been c onsidered and addressed as such. The comments will not be assigned an RFC number by EPA.

Consistent with the statement of the commenter (3747.1), Section 8.5 of EPA’s Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and In tegrity of Information Disseminated by the Environmental Protection Agency, describes that if a group or an individual raises a question regarding information supporting a proposed rule, EPA generally e xpects to treat it procedurally like a comment to the rulemaking, addressing it in the response to co mments rather than through a separate response

5 Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Inte grity of Information Disseminated by the Environmental Protection Agency. EPA/260R-02-008, October 2002.



### 53mechanism. When EPA provides opportunities fo r public participati on by seeking comments on information, the public comment process will genera lly address concerns about EPA’s information. EPA believes that the thorough consideration provided by the public comment process for these Findings serves the purposes of the Guidelines, provides an oppor tunity for correction of any information that does not comply with the Guidelines, and does not duplicate or interfere with the orderly conduct of the action.

After careful review, EPA concluded that it is most appropriate to treat these 13 separate topics of concern as public comments and fully address them in the relevant volumes of the Response to Comments document. In many cases, other commenter s raised analogous points and we have grouped these similar comments, where appropriate, for purposes of our response. Please see the following volumes of this Response to Comments document for our responses to these requests for correction:

R equest for Correction: Peer Review This section (1.5)
R equest for Correction: Treatment of Uncer tainty and Ranges Volume 1: Section 2
R equest for Correction: Selection of Cited Studies This section (1.5)
R equest for Correction: Transparency of Available Data This section (1.5)
R equest for Correction: Transparency in Observational Data This section (1.5)
R equest for Correction: Transparency in Model Limitations Volume 4
R equest for Correction: Baseline Assumption Errors Volume 1: Section 2 Volume 3
R equest for Correction: Adaptation/Mitigation Volume 1: Section 3
R equest for Correction: Defining Likelihood Volume 1: Section 2
R equest for Correction: Misapplication of IPCC Storylines Volume 4
R equest for Correction: Availability of Data This section (1.5)
R equest for Correction: Precipitation Volume 2 Volume 7
R equest for Correction: Sea Level Rise Volume 2

The process used by EPA regarding the information that forms the basis for the Findings maximizes quality, utility, objectivity, and integrity of the inform ation used. See Section III.A. of the Findings, “The Science on Which the Decisions Are Based,” for our response to comments on the use of the assessment literature. Therefore, and as further explained below, we have addressed these topics of concern consistent with EPA’s Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by the Environmental Protection Agency

.

Comment (1-62) : A number of commenters (0289, 3332.1, 3394.1, 3415.1, 3509.1, 3577.1, 3747.1, 7029, 10345, 10524, 10956, 11358.1, and 11466.1) argue that EPA did not make publically available the data, models, and other relevant information used in the studies upo n which the endangerment determination was made. Several commenters (3577.1, 3747.1, and 10071.2) state that nearly half of the studies on which EPA relies for the Proposed Findings are unavailable in the docket due to copyright issues.

Response (1-62) : EPA is required to docket only the information on which it relies, and as explained Section III.A. of the Findings, and in this volume of the Response to Comments document, the Administrator is reasonably relying on the major assessments of the USGCRP, IPCC, and NRC as the primary scientific and technical basis of her endangerment decision. See, e.g., American Trucking Ass’n v. EPA, 293 F.3d 355, 372 (D.C. Cir. 2002) (“ATA III”) (EPA is not required to obta in and publicize the data underlying all the studies on which they rely.). See Section III.A. of the Findings , “The Science on Which the Decisions Are Based,”



### 54for our response to comments on the use of the assessment literature. Further, as noted previously, Section 1(b) and Box 1.1 of the TSD provide a general d escription of the procedures followed by the IPCC and USGCRP/CCSP in developing their reports, and our responses to specific comments on these issues are provided in Section 1.1 of Volume 1 (this volume) of the Response to Comments document.

Where EPA used or described any data in the TSD , for example the GHG emissions data, the source of the data is listed, including where it can be accessed and downloaded. Further, each report, study, or dataset we relied upon has been placed in the Docket 6 for this Action. Information regarding the underlying data, models, and studies used by IPCC, USGCRP, CCSP, and NRC in developing their assessment reports can be accessed by consulting these reports. Given that some of these reports relied upon thousands of underlying studies, supplying every underlying study in the Docket would be unreasonable and unnecessary.

With respect to the comment regarding the reliance of copyrighted materials, we acknowledge that more than half of the scientific literature that EPA re ferences in the TSD has copyright protections and is therefore unavailable for download from the Docket at regulations.gov. However, the public can review all documents used by EPA in developing the TSD by requesting a copy or visiting EPA’s Air Docket. EPA practice regarding the availability of copyrighted ma terials is consistent with law. The online Docket provides the public with an explanation of why certain copyrighted material is not available for download and provides information on how to receive copies of the copyright protected material. The following message is posted for each copyrighted publication in the Docket:

This document is not available in Regulations.gov since it is a copyrighted publication and may not be reproduced without consent of the copyright holder. Contact the EPA Docket Center, Public Reading Room to view or receive a copy of this document. Requests for copies may be made as follows:

In person/writing : Environmental Protection Agency, Docket Center 1301 Constitution Ave NW, 2822T, Room 3334 Washington, DC. 20004

Telephone : 202-566-1744

Fax : 202-566- 9744

Email : docket-customerservice@epa.gov

All sources of information from the assessment reports are clearly cited in the text of the TSD and listed in the references section. This approach ensures the high level of transparency and consistency that is encouraged by EPA’s Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by the Environmental Protection Agency , and will ensure “the reproducibility of such information by qualified third parties, to an acceptable degree of imprecision.”

Comment (1-63) :

6 EPA Docket ID # EPA-HQ-OAR-2009-0171.



### 55A commenter (3747.1) argues that EPA did not adhere to information quality guidelines under the IQA because it did not provide the following regarding the data and models used in the TSD:

R aw data sets used in the analysis of the observational record.
A lgorithms used to correct, transform, or otherwise modify observational datasets.
M issing data algorithms for all datasets.
D atasets used to calibrate models.
M odel calibration results for models.
A djustments made to models as a result of calibration.

Response (1-63) : As previously noted, the IQA does not impose its ow n standards or establish any requirements regarding the quality of agency information; instead, it requir es only that an agency “i ssue guidelines” ensuring data quality. EPA complied with the IQA by issuing its Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by the Environmental Protection Agency and has acted consistently with these Gu idelines in developing these Findings.

As noted above, EPA is required to docket only the information on which it relies, and the Administrator is reasonably relying on the major assessments of the USGCRP, IPCC, and NRC as the primary scientific and technical basis of her endangerment decision. EPA did not conduct new research or modeling in developing the TSD, and instead re lied upon the findings of the assessment literature, including data and modeling studies presented in those reports. The information mentioned by the commenter can be accessed by consulting these assessment reports and the underlying studies. For example, Lemke et al. (2007) describes that “The premier data set used to evaluate large-scale snow covered area (SCA), which dates to 1966 and is the longest satellite-derived environmental data set of any kind, is the weekly visible wavelength satellite maps of NH [Northern Hemis phere] snow cover produced by the US National Oceanic and Atmospheric Administration’s (NOAA) National Environmenta l Satellite Data and Information Service (NESDIA; Robinson et al., 1993).” The Guidelines state: “For disseminated influential original and supporting data, EPA intends to ensure reproducibility according to commonly accepted scientific, financial, or statistical standards. It is important that analytic results for influential information have a higher degree of transparency regard ing (1) the source of the data used, (2) the various assumptions employed, (3) the analytic methods applie d, and (4) the statistical procedures employed.” Our comprehensive referencing of the assessment literatu re ensures transparency regarding the source of the data used and the TSD clearly indicates assump tions (e.g., emissions scenarios), analytical methods, and statistical procedures where that informati on was necessary in describing the conclusions.

Comment (1-64) : A number of commenters (0169, 2972.1, 3187.1, 3286.1, 3297.1, 3307.1, 3332.1, 3347.1, 3372.1, 3394.1, 3415.1, 3481.1, 3579.1, 3596.1, 3705.1, 3722, 3747.1, 3748.1, 4446, 4766, and 9970) argue that EPA did not following IQA guidelines when it decided not to conduct a new analysis of the best available scientific information and instead relied upon IPCC and CCSP assessments.

Response (1-64) : See Section III.A. of the Findings, “The Science on Wh ich the Decisions Are Based,” and Section 1.1 of this volume of the Response to Comments document, for our responses on the use of the assessment literature and EPA’s decision not to conduct an additional and separate review of the underlying climate data and research.



### 56EPA has determined that the approach taken provide d the high level of transparency and consistency outlined by EPA’s Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by th e Environmental Protection Agency. The Guidelines state: “Ensuring and maximizing the quality of information from States, other governments, and third parties is a complex undertaking, involving thoughtful collaboration with States, Tribes, the scientific and technical community, and other external information providers. EPA will continue to take steps to ensure that the quality and transparency of information provided by external sources are sufficient for the intended use.” Our approach is consistent with these guidelines because we thoroughly reviewed and evaluated the author selection, report preparation, expert revi ew, public review, information quality, and approval procedures of IPCC, USGCRP/CCSP, and NRC to ensure the information adhered “to a basic standard of quality, including objectivity, utility, and integrit y.” See responses to comments in Section 1 and Appendices A, B, and C of this Volume for additional information regarding the steps EPA took to ensure that the quality and transparency of information provided by these asse ssments. See Section III.A of the Findings for our rationale for why the assessment lite rature represents the best available scientific information regarding climate change.

Comment (1-65) : A commenter (3747.1) argues that they are unable to evaluate the strength of EPA’s analysis of observational data because the TSD does not describe th e “current state of observational data in detail, including uncertainties in data collection, data assemb ly, data processing, correction factors, and data set concatenations.” The commenter argues that EPA di d not conform to information quality guidelines.

Response (1-65) : As previously noted, the IQA does not impose its ow n standards or establish any requirements regarding the quality of agency information; instead, it requir es only that an agency “i ssue guidelines” ensuring data quality. EPA complied with the IQA by issuing its Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by the Environmental Protection Agency and has acted consistently with these Gu idelines in developing these Findings.

In developing the TSD, EPA primarily relied upon th e findings of the assessment literature, including observational data and studies describing observed c limate change impacts. We did not develop new observational data as part of this action, but rath er summarized existing information from the assessment literature. The specific information regarding observa tional data mentioned by the commenter can be accessed by consulting these assessment reports and th e underlying studies. This approach is consistent with EPA’s Guidelines

.

See Section III.A., “The Science on Which the Decisions Are Based,” and Section 1.1 of this volume of the Response to Comments document, for our res ponses on the use of the assessment literature, our evaluation of the procedures for developing the IPCC and other assessment reports, the treatment of new and additional scientific studies received through the public comment process, and EPA’s decision not to conduct an additional and separate review of the underl ying climate data and research. See Section 1.2 of this volume for EPA’s response to comments on the communication of uncertainties in the underlying assessment literature.

Comment (1-66) : A commenter (11466.1) describes that surface temperat ure data from the University of East Anglia’s Climate Research Unit (CRU) was used by EPA in th e TSD and that the raw data no longer exist, as only the quality-controlled and homogenized data was retained by CRU. The commenter argues that use of this



### 57data violates EPA’s information qua lity guidelines, as the public is unable to access, assess, and verify the data.

Response (1-66) : EPA’s response to this comment is provided in Volu me 2: Section 2 of this Response to Comments document.

Comment (1-67) : A number of commenters (0169, 0797, 2972.1, 3053, 3136.1, 3187.1, 3187.4, 3286.1, 3297.1, 3307.1, 3332.1, 3347.1, 3372.1, 3394.1, 3415.1, 3481.1, 3509.1, 3596.1, 3597.1, 3679.1, 3705.1, 3722, 3747.1, 3748.1, 3764.1, 3769.1, 4766, 10388) argue that EPA di d not adhere to information quality guidelines in using IPCC, CCSP, and NRC reports as the basis of the endangerment finding. Most of these comments argue that EPA did not consider all relevant informat ion before proposing the findings and failed to be transparent regarding the Agency's process for evaluating and considering scientific literature in developing the Proposed Findings. Commenter 3747.1 argues that EPA should more clearly describe the process used in selecting studies and information which were cited in the TSD.

Response (1-67) : EPA has clearly explained how it evaluated and consid ered scientific literature, and therefore has adhered to the information quality guidelines, as further expl ained in the Findings and the Response to Comments document. See Section III.A. of the Findings, “The Science on Which the Decisions Are Based,” and Section 1.1 of this volume of th e Response to Comments document, for our responses on the use of the assessment literature as scientific basis in the TSD, our evaluation of the procedures used by IPCC, USGCRP, CCSP, and NRC in developing the assessment reports, the treatment of new and additional scientific studies received through the public comment process, and EPA’s decision not to conduct an additional and separate review of the underlying c limate data and research . See Section IV of the Findings for detailed discussion of how the Administrato r reached her determination. This section of the Findings was substantially expanded in response to comments.

Comment (1-68) : Several commenters (0322, 3286.1, 3297.1, 3330.1, 3379.1, 3394.1, 3415.1, 3705.1, and 3764.1) argue that EPA has not shown that the IPCC's informati on quality process is equivalent to requirements under the IQA.

Response (1-68) : As previously noted, the IQA does not impose its ow n standards or establish any requirements regarding the quality of agency information; instead, it requir es only that an agency “i ssue guidelines” ensuring data quality. EPA complied with the IQA by issuing its Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by the Environmental Protection Agency and has acted consistently with these Gu idelines in developing these Findings.

See Section III.A. of the Findings, “The Science on Which the Decisions Are Based,” for our responses on the use of the assessment literatu re and our evaluation of the procedures used by IPCC, USGCRP, CCSP, and NRC in developing the assessment reports . As thoroughly discussed in our response to comment 1-14, the IPCC process for preparation and review of its reports demonstrate a formal and sustained commitment to transparency and rigor in report development, review, and acceptance and approval. The assessment reports we used in develo ping the TSD represent the best available science and



### 58supporting studies conducted in accordance with sound and objective scientific practices, are peer reviewed, and adhere to standards of quality based on objectivity, u tility, and integrity.

Comment (1-69) : Several commenters (2972.1, 3136.1, 3307.1, 3679.1, 3747.1, 3748.1, and 3764.1) argue that EPA failed to evaluate and incorporate climate science studies mo re recent than those employed by IPCC and CCSP.

Response (1-69) : The commenters’ assertion is incorrect. See Section III. A. of the Findings, “The Science on Which the Decisions Are Based,” for our response to comments on the use of the assessment literature and Section 1.1 of this Volume for our responses to comment rega rding our treatment of new and additional scientific literature provided through the public comment proce ss. For EPA’s responses to comments and literature provided on specific climate science issues in the TSD, please refer to the appropriate Response to Comment volumes.

Comment (1-70) : Several commenters (3187.1, 3332.1, 3372.1, 3394.1, 3415.1, 3509.1, 3577.1, and 3747.1) argue that EPA did not to utilize an independent and objective peer review process a nd therefore failed to adhere to information quality guidelines under the IQA. Commenter s argue that using federal experts involved with the development of IPCC, USGCRP, CCSP, and NRC reports to review the TSD was inappropriate because they do not represent an objective, independent perspective. One commenter argues that EPA did not describe how the federal experts were c hosen and why this pr ocess was appropriate.

Response (1-70) : Please refer to Section 1.1 of this volume for EPA’s response to comments regarding the review of the TSD, including the role of the federal expert review and why these reviewers were chosen. The ‘Peer Review Policy’ section (4.2) of the Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Inform ation Disseminated by the Environmental Protection Agency states:

EPA's Peer Review Policy provides that majo r scientifically and technically based work products (including scientific, engineering, economic, or statistical documents) related to Agency decisions should be peer-reviewed. Agency managers within Headquarters, Regions, laboratories, and field offices dete rmine and are accountable for the decision whether to employ peer review in particular instances and, if so, its character, scope, and timing. These decisions are made consistent with program goals and priorities, resource constraints, and statutory or court-ordered deadlines. For those work products that are intended to support the most important decisions or that have special importance in their own right, external peer review is the procedure of choice.

Given our approach to the scientific literature (describ ed in Section III.A of the Findings), the purpose of the federal expert review was to ensure that the TSD accurately summarized the conclusions and associated uncertainties from the assessment reports. This peer-review approach for the TSD is consistent with EPA’s Guidelines because: 1) the reviewers contributed significantly to the body of climate change literature and played active roles in IPCC and CC SP—therefore making them experts on various aspects of climate science and very familiar with the underlying literature and state of the science, 2) the federal climate change experts represent a range of technical specialties that span the range of topics covered in the TSD and covered by the range of topics that the Administrator needed to consider, 3) the federal experts offered unbiased and objective review because th ey were external and therefore not involved with



### 59the development of the TSD 7 , and 4) the TSD clearly identifies the Federal expert reviewers (as it did in the version of the TSD for the Proposed Findings), which ensures transparency.

Comment (1-71) : Several commenters (3187.4, 3307.1, 3372.1, 3729.1, and 3764.1) argue that EPA dismissed significant comments, criticisms, or suggested data/studies received through the ANPR without demonstrating why they were not considered.

Response (1-71) : Please see Section 1.1 of this volume for response to comments on how EPA reviewed and considered public comments submitted on the ANPR.

Comment (1-72) : Two commenters (3136.1 and 3446.2) argue that th e USGCRP's assessment report “Global Climate Change Impacts in the United States” does not comply with the IQA. A commenter (3347.1) argues that EPA failed to describe how the CCSP ensured that th e 21 synthesis and assessment products, rather than the documents on which it relied, satisfied EPA's IQA guidelines.

Response (1-72) : Please see Section 1.1 of this volume for response to comments regarding the procedures that governed development of the USGCRP and CCSP reports. Th e foundation of the USGCRP’s June 2009 assessment report is the set of 21 CCSP SAPs, which were designe d to address key policy-relevant issues in climate science. All 21 SAPs met federal requirements pertaining to the IQA. Furthermore, the unified synthesis report was extensively reviewed and revised based on comments from experts and the public. The report was approved by its lead USGCRP agency, NOAA, the other USGCRP agencies, and the Committee on the Environment and Natural Resources on behalf of the National Science and Technology Council. The unified synthesis product complies with the IQA a nd is consistent with EPA’s information quality guidelines.

Regarding the information quality procedures for the individual SAPs, the CCSP provided strict guidance for the development and review of the 21 SAPs. The ‘Information Quality Procedures’ section of the Guidance to Agency Leads Regarding the Prepara tion of CCSP Synthesis and Assessment Products

states:

As a lead agency for an SAP, you will be r esponsible for complying with the Information Quality Act. Section 515 of Public Law 106-554, known as the Data Quality Act, required the Office of Management and Budget to promulgate guidance to agencies ensuring the quality, objectivity, utility, and integrity of information (including statistical information) disseminated by Federal ag encies. OMB's government-wide guidelines, published as interim final on September 28, 2002 (66 F.R. 49718) and finalized on February 22, 2002 (67 F.R. 8452), can be found on OMB’s OIRA's Web site http://www.whitehouse.gov/omb/inforeg/agency _i nfo_qualit y_links.html

.
To facilitate any requests that might ensue pursuant to this statute, it is a good idea that you advise each of your authors to:

7 We note that 11 of the 12 federal ex pert reviewers do not work at EPA. Th e one EPA reviewer was not involved in the development of the TSD and therefore was an objective, unbiased reviewer.



### 60a. Retain a copy of any document they cite; b. Identify specific pages in the literature they cite, where appropriate; and c. Use explanatory footnotes when the te xt extends beyond a fact cited in the literature.

Each product team should also be prepared to make available any data it uses in the report’s charts or figures. These data should be publicly available as early as the expert review and public comment phases and should remain available thereafter.

The National Science and Technology Council, a Ca binet-level council that coordinates science and technology research across the federal government, reviewed and approved each report, including the steps taken by the lead agency to ensure that its information quality guidelines were followed. Further information clearance, review, and data quality procedures for the CCSP SAPs can be found in Appendix B. This process was robust, objective, transparent, and complete and ensured that the USGCRP and CCSP reports summarized in the TSD were consistent with the IQA Guidelines

.

Comment (1-73) : Several commenters (3509.1, 3577.1, 3596.1, 3747.1, 3748.1, 3764.1, 3769.1, and 11358.1) argue that EPA did not discuss or reference scientific studies th at do not support the Agen cy's conclusion that GHG emissions endanger human health and welfare.

Response (1-73) : Please see Section 1.1 of this volume for responses to comments regarding EPA’s consideration of studies and information, including those which offered a lternative views to the findings of the assessment literature.

Comment (1-74) : Two commenters (3397 and 3415.1) suggest that EPA ensure that it complies with the IQA and similar EPA guidelines in future actions re garding climate change policy.

Response (1-74) : As noted above, EPA has complied with the IQA by issuing guidelines that ensure and maximize the quality, objectivity, utility, and integrity of the info rmation the Agency disseminates. In developing the Proposal, the TSD, and this Action, EPA developed an d prepared the information in a manner that is consistent with EPA's Guidelines for Ensuring and Maximizi ng the Quality, Objectivity, Utility and Integrity of Information Disseminated by the Environmental Protection Agency . EPA intends to continue to act consistently with the IQA and its guidelines in future actions.

Comment (1-75) : Two commenters (3347.1 and 3764.1) suggest that in issuing a final finding, EPA employ information and data quality standards pursuant to tho se under the Safe Drinking Water Act (SDWA).

Response (1-75) : Unlike the IQA, the 1996 Amendments to the SDWA 8 added specific information requirements for decisions under SDWA based on science that is used to establish federal drinking water standards for

8 Safe Drinking Water Act Amendments of 1996, 42 U.S.C. 300g-1(b)(3)(A) & (B).



### 61public water systems. The information quality standa rds of the SWDA Amendments were adapted into EPA’s Guidelines for Ensuring and Maximizing the Q uality, Objectivity, Utility and Integrity of Information Disseminated by the Environmental Protection Agency. The SDWA obligations do not apply to this action. However, EPA has acted cons istently with its information quality guidelines

. Section 6.4 of EPA’s Guidelines states that:

In our dissemination of influential scientific information regarding human health, safety or environmental risk assessments, EPA will ensure, to the extent practicable and consistent with Agency statutes and existing legislative regulations, the objectivity of such information disseminated by the Agency by applying the following adaptation of the quality principles found in the Safe Drinking Water Act (SDWA) Amendments of 1996:

(A) The substance of the information is accu rate, reliable and unbiased. This involves the use of: (i) The best available science and suppor ting studies conducted in accordance with sound and objective scientific practices, including, when available, peer reviewed science and supporting studies; and (ii) Data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies the use of the data).

(B) The presentation of information on human health, safety, or environmental risks, consistent with the purpose of the informa tion, is comprehensive, informative, and understandable. In a document made available to the public, EPA specifies: (i) Each population addressed by any estimate of applicable human health risk or each risk assessment endpoint, including populations if applicable, addressed by any estimate of applicable ecological risk; (ii) The expected risk or central estim ate of human health risk for the specific populations affected or the ecological assessment endpoints, including populations if applicable; (iii) Each appropriate upper-bound or lower-bound estimate of risk; (iv) Each significant uncertainty identifie d in the process of the assessment of risk and studies that would assist in resolving the uncertainty; and (v) Peer-reviewed studies known to the Administrator that support, are directly relevant to, or fail to support any estim ate of risk and the methodology used to reconcile inconsistencies in the scientific data.

EPA followed has followed its information quality gui delines in developing the Findings. We relied on the best available peer-reviewed science, suppor ting studies conducted in accordance with sound and objective scientific practices, and data collected by accepted or best available methods. The Agency also has presented the information in a manner that is comprehensive, informative, and understandable.



### 62References

American Association for the Advancement of Scien ce (2009). A letter from many scientific societies to U.S. senators on climate change legislation. October 21. http://www.ametsoc.org/sss/documents/climateletter.pdf . Accessed December 5, 2009.

Carlin, A. (2009). Proposed NCEE Comments on Draft Technical Support Document for Endangerment Analysis for Greenhouse Gas Emissions Under the Clean Air Act.

CCSP (2009). Best practice approaches for characterizi ng, communicating, and incorporating scientific uncertainty in decisionmaking. [Granger Morgan, H. Do wlatabadi, M. Henrion, D. Keith, R. Lempert, S. McBrid, M. Small, T. Wilbanks (eds.)]. W ashington, DC: National Oceanic and Atmospheric Administration.

CCSP (2007). Clarification of Review and Clearance Process for CCSP Synthesis and Assessment Products. Memorandum dated 3 August 2007 to CENR and CCSP Principals from William Brennan, Acting Director, U.S. Climate Change Science Program. http://www.climatescience.gov/Library/sap/sapguidelines-clarification-aug2007.htm . Accessed November 30, 2009.

CCSP (2006). Guidance to Agency Leads Regarding the Pre paration of CCSP Synthesis and Assessment Products . Washington, DC.

CCSP (2004). Guidelines for Producing CCSP Synthesis and Assessment Products . Washington, DC. http://www.climatescience.gov/Library/sap/sap-guidelines.pdf . Accessed November 30, 2009.

Center for Inquiry (2009). Ranking member’s Senate minority report on global warming not credible, says CFI. http://www.centerforinquiry.net/opp /ne ws/senat e_minority_report_on_global_warming_not_credible/

.
Cruz, R. et al. (2007). Asia. In: M.L. Parry, O.F. Ca nziani, J.P. Palutikof, P.J. van der Linden, and C.E. Hanson, eds. Climate Change 2007: Impacts, Adaptation, and Vulnerability. Contribution of Working Group II to the Fourth Assessment Report of th e Intergovernmental Panel on Climate Change. Cambridge, United Kingdom, and New York, NY: Cambridge University Press.

Davis, R.E., P.C. Knappenberger, P.J. Michaels, and W.M. Novicoff (2003a). Changing heat-related mortality in the United States. Environmental Health Perspectives 111(14):1712-1718.

Doran, P.T., M.K. Zimmerman (2009). Examining the scientific consensus on climate change. EOS 90:22-23.

Forster, P., et al. (2007). Changes in atmosphe ric constituents and in radiative forcing. In: Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change.

Goklany, I.M. (2006). Death and Death Rates Due to Extreme Weather Events: Global and U.S. Trends, 1900-2004 . Prepared for the proceedings of the C limate Change & Disaster Losses Workshop, Hohenkammer, Germany, May 25-26.

ICSC (2008). The Manhattan Declaration: Global warming is not a global crisis. March 4. http://www.climatescienceinternational.org/index.php?option=com

_ content&task =view&id=37&Itemid=
1 . Accessed December 5, 2009.



### 63Idso, C., and S.F. Singer (2009). Climate Change Reconsidered: 2009 Report of the Nongovernmental Panel on Climate Change (NIPCC). Chicago, IL: The Heartland Institute.

IPCC (2007) Fourth Assessment Report: Climate Change 2007. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA.

IPCC (2006). Principles Governing IPCC Work . Approved at the Fourteenth Session (Vienna, 1-3 October 1998) on 1 October 1998, amended at the 21 st Session (Vienna, 3 and 6-7 November 2003) and at the 25th Session (Mauritius, 26-28 April 2006). http://www.ipcc.ch/pdf/ipcc-principles/ipccprinciples.pdf . Accessed November 30, 2009.

IPCC (2005). Guidance Notes for Lead Authors of the IPCC Fourth Assessment Report on Addressing
Uncertainties. http://www.ipcc.ch/pdf/assessment-report/a r4/wg1/ar4-uncertaintyguidancenot e.pdf

. Accessed November 30, 2009.

IPCC (2004). Guidance Notes for Lead Authors of the IPCC Fourth Assessment Report on Addressing
Uncertainties. http://www.ipcc.ch/pdf/assessment-report/a r4/wg1/ar4-uncertaintyguidancenot e.pdf

. Accessed November 30, 2009.

IPCC (2001) Third Assessment Report: Climate Change 2001. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA.

IPCC (1999). Procedures for the Preparation, Review, Accep tance, Adoption, Approval, and Publication of IPCC Reports. Adopted at the Fifteenth Session of the IPCC (San Jose, 15-18 April 1999) and revised, on a provisional basis, at the Twentieth Sessi on of the IPCC (Paris, 19-21 February 2003). http://www.climatescience.gov/Library/ipcc/app-a.pdf . Accessed November 30, 2009.

Manning, M.R., M. Petit, D. Easterling, J. Murphy, A. Patwardhan, H-H. Rogner, R. Swart, and G. Yohe, eds. (2004). IPCC Workshop on Describing Scientific Uncertainties in Climate Change to Support Analysis of Risk and of Options: Workshop report . Geneva, Switzerland: In tergovernmental Panel on Climate Change.

Monckton, C.W. (2009a). Testimony of the Ri ght Honorable Christopher Walter Monckton, Third Viscount Monckton of Brenchley, before the En ergy and Commerce Committee of the House of Representatives on March 25, 2009.

Monckton, C.W. (2009b). The Viscount Monckton of Brenchley's March 30, 2009 letter to Representatives Markey and Barton, http://scienceandpublicpolicy.org/images/stories/ papers/reprint/m arke y_and_barton_letter.pdf . Accessed December 5, 2009.

Moss, R., and S. Schneider (2000). Uncertainties. In: R. Pachauri, T. Taniguchi, and K. Tanaka, eds. , in Guidance Papers on the Cross Cutting Issues of the Third Assessment Report of the IPCC . Geneva, Switzerland, Intergovernmental Panel on Climate Change..

Muir-Wood, R., S. Miller, and A. Boissonade (200 6). The search for trends in a global catalogue of normalized weather-related catastrophe losses. Work shop on climate change and disaster losses: Understanding and attributing tr ends and projections. Final Workshop Report, 25-26 May, Hohenkammer, Germany. http://sciencepolicy.colorado.edu/sparc/research/pr ojects/extreme_events/ m unich_workshop/workshop_re port.html . Accessed November 30, 2009.



### 64

### 65 Munich Re Group (2000). Topics, 2000: Natural Catastrophes, The Current Position. Munich, Germany. http://www.munichre.com/publications/302-02354_en.pdf . Accessed November 30, 2009.

National Academies (2008). Report Review Committee: Guidelines for Review of Consensus Reports. Washington, DC. http://sites.nationalacademies.org/NRC/PoliciesandProcedures/NRC_039341 . Accessed November 30, 2009.

National Academies (2006). Our Study Process: Ensuring Independent, Objective Advice. Washington, DC. http://sites.nationalacademies.org/NRC/PoliciesandProcedures/index.htm . Accessed November 30, 2009.

National Academy of Sciences (2008). Understanding and Responding to Climate Change: 2008 Edition. Washington, DC. http://dels.nas.edu/dels/rpt_brie fs/climate_change_2008_final. p df . Accessed November 30, 2009.

National Academies (2005). Getting to Know the Committee Process. Washington, DC. http://sites.nationalacademies.org/N RC/ PoliciesandProcedures/index.htm . Accessed November 30, 2009.

Office of Climate and Energy, State of New Jersey (2008). Global Warming Response Act Recommendations Report. Draft. December 15.

Office of the Press Secretary (March 9, 2009) Scie ntific Integrity: Memorandum for the Head of Executive Departments and Ag encies, The White House.

Peterson, T.C., W.M. Connelley, and J. Fleck (2008). The myth of the 1970s global cooling scientific

consensus. Bulletin of the American Meteorological Society 89:1325-1337.

Presidential/Congressional Commission on Risk Assessment and Risk Management (1997). Framework for Environmental Health Risk Management . Final Report, Volume 1.

Robinson, A.B., N.E. Robinson, and W. Soon (2007) . Environmental effects of increased atmospheric carbon dioxide. Journal of American Physicians and Surgeons 12:79-90.

U.S. EPA (2009). Inventory of U.S. Greenhouse Gas Emissions and Sinks . Washington, DC: U.S. Environmental Protection Agency. EPA/430/R-09/004.

U.S. EPA (2002). Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity, of Information Disseminated by th e Environmental Protection Agency. Washington, DC: U.S. Environmental Protection Agency. EPA/260/R-02/008. http://www.epa.gov/QUALITY/informationguidelin es/docu m ents/EPA_InfoQualityGuidelines.pdf

. Accessed November 30, 2009.

U.S. Senate Minority Report Update Report (2009). http://epw.senate.gov/public/index.cfm?FuseAction= Minorit y .Blogs&ContentRecord_id=2674e64f-802a23ad-490b-bd9faf4dcdb7 Accessed November 30, 2009.

## Appendix A IPCC Principles and Procedures

1) Principles Governing IPCC Work

2) Procedures for the Preparation, Review, Acceptance, Adoption, Approval, and Publication of IPCC Reports

3) Guidance Notes for Lead Authors of the IPCC Fourth Assessment Report on Addressing Uncertainties 6 7

69

8 4

66PRINCIPLES GOVERNING IPCC WORK

Approved at the Fourteenth Session (Vienna, 1

- 3 October 1998) on 1 October 1998, amended at the 21 st Session (Vienna, 3 and 6

- 7 November 2003) and at the 25 th Session (Mauritius, 26

- 28 April 2006)

________________________ _________________________________________________________

INTRODUCTION

1. The Intergovernmental Panel on Climate Change (hereinafter referred to as the IPCC or, synonymously, the Panel) shall concentrate its activities on the tasks allotted to it by the relevant WMO Executive Council and UNEP Governing Council resolutions and decisions as well as on actions in support of the UN Framework Convention on Climate Change process.

ROLE

2. The role of the IPCC is to assess on a comprehensive, objective, open and transparent basis the scientific, technical and socio

- economic information relevant to understanding the scientific basis of risk of human

- induced climate change, its potential impacts and options for adaptation and mitigation. IPCC reports shoul

d be neutral with respect to policy, although they may need to deal objectively with scientific, technical and socio

- economic factors relevant to the application of particular policies.

3. Review is an essential part of the IPCC process. Since the IPCC is an intergovernmental body, review of IPCC documents should involve both peer review by experts and review by governments.

ORGANIZATION

4. Major decisions of the IPCC will be taken by the Panel in plenary meetings.

5. The IPCC Bureau, the IPCC Working G roup Bureaux and the Bureaux of any Task Forces of the IPCC shall reflect balanced geographic representation with due consideration for scientific and technical requirements.

6. IPCC Working Groups and any Task Forces constituted by the IPCC shall have cl early defined and approved mandates and work plans as established by the Panel, and shall be open

- ended.

PARTICIPATION

7. Participation in the work of the IPCC is open to all UNEP and WMO Member countries.

8. Invitations to participate in the sessions o f the Panel and its Working Groups, Task Forces and IPCC workshops shall be extended to Governments and other bodies by the Chairman of the IPCC.

9. Experts from WMO/UNEP Member countries or international, intergovernmental or non

governmental organisatio ns may be invited in their own right to contribute to the work of the IPCC Working Groups and Task Forces. Governments should be informed in advance of invitations extended to experts from their countries and they may nominate additional experts.

67PROCEDU RES

10. In taking decisions, and approving, adopting and accepting reports, the Panel, its Working Groups and any Task Forces shall use all best endeavours to reach consensus. If consensus is judged by the relevant body not possible: (a) for decisions on procedural issues, these shall be decided according to the General Regulations of the WMO; (b) for approval, adoption and acceptance of reports, differing views shall be explained and, upon request, recorded. Differing views on matters of a scientific,

tec hnical or socio

- economic nature shall, as appropriate in the context, be represented in the scientific, technical or socio

- economic document concerned. Differences of views on matters of policy or procedure shall, as appropriate in the context, be recorded in the Report of the Session.

11. Conclusions drawn by IPCC Working Groups and any Task Forces are not official IPCC views until they have been accepted by the Panel in a plenary meeting.

12. Invitations to participate in the sessions of the Panel and its Working Groups, Task Forces and IPCC workshops should be extended at least six weeks in advance of the opening of the session.

13. Major reports, including Assessment Reports, Special Reports and Methodology Reports, basic documentation and other avai lable reports for consideration at the sessions of the Panel and its Working Groups shall normally be made available by the IPCC Secretariat at least four weeks in advance of the session and, to the extent possible, in all official UN languages.

14. Inte

rpretation into all official UN languages shall be provided for all sessions of the IPCC meeting in plenary, of its Bureau and its Working Groups.

15. The scheduling of the sessions of the Panel and its Working Groups and Task Forces shall be

co

- ordinate

d, to the extent possible, with other related international meetings.

16. These Principles shall be reviewed at least every five years and amended as appropriate.

17. Procedures for the preparation, review, acceptance, approval, adoption and publication of IPCC reports are given in Appendix A.

18. Financial procedures for the IPCC are given in Appendix B.

19. Rules and Procedures for the Election of the IPCC Bureau and Any Task Force Bureau are given in Appendix C.

68

### PROCEDURES FOR THE PREPARATION, REVIEW, ACCEPTANCE, ADOPTION, APPROVAL AND PUBLICATION OF IPCC REPORTS Adopted at the Fifteenth Session (San Jose, 1 5

- 18 April 1999) amended at the Twentieth Session (Paris, 19-21 February 2003) and Twenty-first Session (Vienna, 3 and 6-7 November 2003)

CONTENTS

1. INTRODUCTION

2. DEFINITIONS

3. IPCC MATERIAL

4. ASSESSMENT REPORTS, SYNTHESIS REPORTS, SPECIAL REPORTS AND METHODOLOGY REPORTS

4.1 Introduction to Review Process

4.2 Reports Accepted

by Working Groups and Reports prepared by the Task Force on National Greenhouse Gas Inventories 4.2.1 Compilation of Lists of Coordinating Lead Authors, Lead Authors, Contributing Authors, Expert Reviewers, Review Editors and Government Focal Points 4.2.2 Selection o

f Lead Authors 4.2.3 Preparation of Draft Report 4.2.4 Review 4.2.4.1 First Review (by Experts) 4.2.4.2 Second Review (by Governments and Experts) 4.2.5 Preparation of Final Draft Report

4.3 Approval and Acceptance of Summaries for Policymakers and Adoption of Overview Chapters of Methodology

Reports

4.4 Reports Approved and/or Adopted by the Panel 4.4.1 The Synthesis Report

5. TECHNICAL PAPERS

6. IPCC SUPPORTING MATERIAL

6.1 Workshops and Expert Meetings

6.2 Co

- sponsored Workshops and Expert Meetings

ANNEX 1 TASKS AND RESPONSIBILITIES FOR LEAD AUTHORS, COORDI NATING LEAD AUTHORS, CONTRIBUTING AUTHORS, EXPERT REVIEWERS AND REVIEW EDITORS OF IPCC REPORTS AND GOVERNMENT FOCAL POINTS

ANNEX 2 PROCEDURES FOR USING NON

- PUBLISHED/NON

-

PEER

- REVIEWED SOURCES IN IPCC REPORTS

69

### 1. INTRODUCTION

This provisionally revised A ppendix to the Principles Governing IPCC Work contains the procedures for the preparation, review, acceptance, adoption, approval and publication of IPCC

r eports and other material s relevant to methodologies

. This Appendix complements the P rocedures for th e P reparation , Review, Acceptance, Adoption , Approval and Publication of IPCC Reports, which was a dopted at the Fifteenth Session of the IPCC (San Jose, 15

- 18 April 1999)

.

2. DEFINITIONS

The definitions of terms used in this document are as follows:

“ acceptance” of IPCC Reports at a Session of the Working Group or Panel signifies that the material has not been subject to line by line discussion and agreement, but nevertheless presents a comprehensive, objective and balanced view of the subject matter. “adoption” of IPCC Reports is a process of endorsement section by section (and not line by line) used for the longer report of the Synthesis Report as described in section 4.3 and for Overview Chapters of Methodology Reports “approval” of IPCC Summaries for Policymakers signifies that the material has been subjected to detailed, line by line discussion and agreement. “Assessment Reports” are published materials composed of the full scientific and technical assessment of climate change, generally in three volumes, one for each of the Working Groups of the IPCC. Each of the volumes may be composed of two or more sections including: (a) a Summary for Policymakers (b) an optional technical summary and (c) individual chapters and their executive summaries. “Mem bers of the IPCC” are countries who are Members of WMO and/or UNEP. “Methodology Reports ” are published materials

, which provide practical guidelines for the preparation of greenhouse gas inventories. Such reports may be composed of two or more sections in cluding: (a) an Overview Chapter, which broadly describes the background, structure and major features of the report, (b) individual chapters and (c) technical Annexes. “Reports” refer to the main IPCC materials (including Assessments, Synthesis, Methodolo gy and Special Reports and their Summaries for Policy Makers and Overview Chapters). “Session of a Working Group” refers to a series of meetings at the plenary level of the governmental representatives to a Working Group of the IPCC. “Session of the Bureau ” refers to a series of meetings of the elected members of the IPCC Bureau who may be accompanied by a representative of their government.

“ Task Force Bureau” refers to the elected members of the Bureau of the Task Force on National Greenhouse Gas Inventor ies. It is chaired by two Co

- chairs, referred to in the following as Task Force Bureau Co

- chairs . “Session of the Panel” refers to a series of meetings at the plenary level of the governmental representatives to the IPCC. “Special Report” is an assessme nt of a specific issue and generally follows the same structure as a volume of an Assessment Report. “Summary for Policymakers” is a component of a Report, such as an Assessment, Special or Synthesis Report, which provides a policy

- relevant but policy

- neut ral summary of that Report. “Supporting Material” consists of published material, workshop proceedings and material from expert meetings which are either commissioned or supported by the IPCC. S u p p o r t i n g m a t e r i a l m a y i n c l u d e s o f t w a r e o r d a t a b a s e s t o f

a c i l i t a t e t h e u s e o f t

h e I P C C M e t

h o d o l o g y R e p o r t s

. “Synthesis Reports” synthesise and integrate materials contained within the Assessment Reports and Special Reports and are written in a non

- technical style suitable for policymakers and address a broad

- range of policy

relevant but policy

- neutral questions. They are composed of two sections as follows: (a) a Summary for Policymakers and (b) a longer report. “Technical Papers” are based on the material already in the Assessment Reports and Special Reports and are prepared on topics for which an objective international scientific/technical perspective is deemed essential.

70

### 3. IPCC MATERIAL

There are three main classes of IPCC materials, each of which is defined in Section 2.

A. IPCC Reports (which include Assessmen ts, Synthesis and Special Reports and their Summaries for Policymakers and Methodology Reports ) B. Technical Papers C. Supporting Materials

The different classes of material are subject as appropriate to different levels of formal endorsement. These levels are described in terms of acceptance, adoption and approval as defined in Section 2.

The different levels of endorsement for the different classes of IPCC material are as follows:

A. In general, IPCC Reports are accepted by the appropriate Working Group. Repor ts prepared by the Task Force on National Greenhouse Gas Inventories are accepted by the Panel. Summaries for Policymakers are approved by the appropriate Working Groups. (Section 4.2) and subsequently accepted by the Panel (Section 4.3). Overview chapters of Methodology Reports are adopted, section by section, by the appropriate Working Group or in case of reports prepared by the Task Force on National Greenhouse Gas Inventories by the Panel. In the case of the Synthesis Report the Panel adopts the underl ying Report, section by section, and approves the Summary for Policymakers. The definition of the terms “acceptance ” , “ adoption” and "approval" will be included in the IPCC published Reports (Section 4.4).

B. Technical Papers are not accepted, approved or ad opted by the Working Groups or the Panel but are finalised in consultation with the Bureau (Section 5)



### C. Supporting Materials are not accepted, approved or adopted (Section 6).

4. ASSESSMENT REPORTS, SYNTHESIS REPORTS, SPECIAL REPORTS AND METHODOLOGY REPO RTS

4.1 Introduction to Review Process

The review process generally takes place in three stages: expert review of IPCC Reports, government/expert review of IPCC Reports, government review of the Summaries for Policymakers , Overview Chapters and/or the S ynthesis Report. Working Group /Task Force Bureau Co

- Chairs should aim to avoid (or at least minimise) the overlap of government review periods for different IPCC Reports and with Sessions of the Conference of Parties of the United Nations Framework Convent ion of Climate Change and its subsidiary bodies.

Expert review should normally be eight weeks, but not less than six weeks, except to the extent decided by the Panel. Government and government/expert reviews should not be less than eight weeks, except to the extent decided by the Panel.

All written expert, and government review comments will be made available to reviewers on request during the review process and will be retained in an open archive in a location determined by the IPCC Secretariat on compl etion of the Report for a period of at least five years.

4.2 Reports Accepted by Working Groups and Reports prepared by the Task Force on National Greenhouse Gas Inventories

Reports presented for acceptance at Sessions of the Working Groups , or in case of reports prepared by the Task Force on National Greenhouse Gas Inventories reports presented for acceptance by the Panel, are the full scientific, technical and socio

- economic Assessment Reports of the Working Groups, Special Reports and

71

### Methodology Repo rts , that is, the IPCC Guidelines for National Greenhouse Gas Inventories or the IPCC

Technical Guidelines for Assessing Climate Change Impacts and Adaptations.

The subject matter of these Reports shall conform to the terms of reference of the relevant Wo rking Groups, or the Task Force on National Greenhouse Gas Inventories and to the work plan approved by the Panel.

Reports to be accepted by the Working Groups, and reports prepared by the Task Force on National Greenhouse Gas Inventories will undergo ex pert and government/expert reviews. The purpose of these reviews is to ensure that the Reports present a comprehensive, objective, and balanced view of the areas they cover. While the large volume and technical detail of this material places practical limi tations upon the extent to which changes to these Reports will normally be made at Sessions of Working Groups or the Panel , "acceptance" signifies the view of the Working Group or the Panel that this purpose has been achieved. The content of the authored c hapters is the responsibility of the Lead Authors, subject to Working Group or Panel acceptance. Changes (other than grammatical or minor editorial changes) made after acceptance by the Working Group or the Panel shall be those necessary to ensure consiste ncy with the Summary for Policymakers or the Overview Chapter . These changes shall be identified by the Lead Authors in writing and made available to the Panel at the time it is asked to accept the Summary for Policymakers , in case of reports prepared by t he Task Force on National Greenhouse Gas Inventories by the end of the session of the Panel which adopts/accepts the report.

Reports accepted by Working Groups, or prepared by the Task Force on National Greenhouse Gas Inventories should be formally and p

rominently described on the front and other introductory covers as:

"A report accepted by Working Group X of the IPCC [- OR

– A report prepared by the Task Force on National Greenhouse Gas Inventories of the IPCC and accepted by the Panel ] but not approved

in detail."

It is essential that Working Group and Task Force on National Greenhouse Gas Inventories work programmes allow enough time in their schedules, according to procedures, for a full review by experts and governments and for the acceptance of th e report . The Working Group /Task Force Bureau Co

- Chairs are responsible for implementing the work programme and ensuring that proper review of the material occurs in a timely manner.

To ensure proper preparation and review, the following steps should be u ndertaken:

1. Compilation of lists of Coordinating Lead Authors, Lead Authors, Contributing Authors, Expert Reviewers, Review Editors and Government Focal Points.
2. Selection of Lead Authors. 3. Preparation of draft Report. 4. Review

. 

### a. First review (by experts).

b. Seco nd review (by governments and experts).

5. Preparation of final draft Report.
6. Acceptance of Report at a Session of the Working Group(s) or the Panel respectively.



### 4.2.1 Compilation of Lists of Coordinating Lead Authors, Lead Authors, Contributing Authors, E xpert Reviewers, Review Editors and Government Focal Points

At the request of Working Group /Task Force Bureau Co

- Chairs through their respective Working Group /Task
Force Bureau, and the IPCC Secretariat, governments, and participating organisations and t he Working Group /Task Force Bureaux should identify appropriate experts for each area in the Report who can act as potential Coordinating Lead Authors, Lead Authors, Contributing Authors, expert reviewers or Review Editors. To facilitate the identification of experts and later review by governments, governments should also designate their respective Focal Points. IPCC Bureau Members and Members of the Task Force Bureau should contribute where necessary to identifying appropriate Coordinating Lead Authors, L ead Authors, Contributing Authors, expert reviewers, and Review Editors in cooperation with the Government Focal Points within their region to

72

### ensure an appropriate representation of experts from developing and developed countries and countries with

econom ies in transition. These should be assembled into lists available to all IPCC Members and maintained by the IPCC Secretariat. The tasks and responsibilities of Coordinating Lead Authors, Lead Authors, Contributing Authors, expert reviewers, Review Editors and government Focal Points are outlined in Annex 1.

4.2.2 Selection of Lead Authors

Coordinating Lead Authors and Lead Authors are selected by the relevant Working Group/ Task Force Bureau, under general guidance and review provided by the Session of the Working Group or, in case of reports prepared by the Task Force on National Greenhouse Gas Inventories, the Panel, from those experts cited in the lists provided by governments and participating organisations, and other experts as appropriate, known throu gh their publications and works. The composition of the group of Coordinating Lead Authors and Lead Authors for a section or chapter of a Report shall reflect the need to aim for a range of views, expertise and geographical representation (ensuring appropr iate representation of experts from developing and developed countries and countries with economies in transition). There should be at least one and normally two or more from developing countries. The Coordinating Lead Authors and Lead Authors selected by the Working Group /Task Force Bureau may enlist other experts as Contributing Authors to assist with the work.

At the earliest opportunity, the IPCC Secretariat should inform all governments and participating organisations who the Coordinating Lead Authors and Lead Authors are for different chapters and indicate the general content area that the person will contribute to the chapter.



### 4.2.3 Preparation of Draft Report

Preparation of the first draft of a Report should be undertaken by Coordinating Lead Aut hors and Lead Authors. Experts who wish to contribute material for consideration in the first draft should submit it directly to the Lead Authors. Contributions should be supported as far as possible with references from the peer

- reviewed and international

ly available literature, and with copies of any unpublished material cited. Clear indications of how to access the latter should be included in the contributions. For material available in electronic format only, a hard copy should be archived and the loca tion where such material may be accessed should be cited.

Lead Authors will work on the basis of these contributions, the peer

- reviewed and internationally

- available literature, including manuscripts that can be made available for IPCC review and selected non

- peer review literature according to Annex 2 and IPCC Supporting Material (see section 6). Material which is not published but which is available to experts and reviewers may be included provided that its inclusion is fully justified in the context of the IPCC assessment process (see Annex 2).

In preparing the first draft, and at subsequent stages of revision after review, Lead Authors should clearly identify disparate views for which there is significant scientific or technical support, together with the relevant arguments. Technical summaries provided will be prepared under the leadership of the Working Group /Task
Force Bureaux.

4.2.4 Review

Three principles governing the review should be borne in mind. First, the best possible scientific and techn ical advice should be included so that the IPCC Reports represent the latest scientific, technical and socio

- economic findings and are as comprehensive as possible. Secondly, a wide circulation process, ensuring representation of independent experts ( i.e. experts not involved in the preparation of that particular chapter) from developing and developed countries and countries with economies in transition should aim to involve as many experts as possible in the IPCC process. Thirdly, the review process shoul

d be objective, open and transparent.

73

### To help ensure that Reports provide a balanced and complete assessment of current information, each Working

Group/ Task Force Bureau

should normally select two Review Editors per chapter (including the executive

summar ies) and per technical summary of each Report.

Review Editors should normally consist of a member of the Working Group /Task Force Bureau, and an independent expert based on the lists provided by governments and participating organisations. Review Editors should not be involved in the preparation or review of material for which they are an editor. In selecting Review Editors, the Bureaux should select from developed and developing countries and from countries with economies in transition, and should aim for a balanced representation of scientific, technical, and socio

- economic views.

4.2.4.1 First Review (by Experts)

First draft Reports should be circulated by Working Group /Task Force Bureau Co

- Chairs for review by experts selected by the Working Group/ Tas k Force Bureaux and, in addition, those on the lists provided by governments and participating organisations, noting the need to aim for a range of views, expertise, and geographical representation. The review circulation should include:

• Experts who have significant expertise and/or publications in particular areas covered by the Report.

• Experts nominated by governments as Coordinating Lead Authors, Lead Authors, contributing authors or expert reviewers as included in lists maintained by the IPCC Secretar iat.

• Expert reviewers nominated by appropriate organisations.

The first draft Reports should be sent to Government Focal Points, for information, along with a list of those to whom the Report has been sent for review in that country.



### The Working Group

/T ask Force Bureau Co

- Chairs should make available to reviewers on request during the review process specific material referenced in the document being reviewed, which is not available in the international published literature.

Expert reviewers should provi de the comments to the appropriate Lead Authors through the relevant Working Group /Task Force Bureau

Co

- Chairs with a copy, if required, to their Government Focal Point.

Coordinating Lead Authors, in consultation with the Review Editors and in coordinati on with the respective Working Group

/Task Force Bureau

Co

- Chairs and the IPCC Secretariat, are encouraged to supplement the draft revision process by organising a wider meeting with principal Contributing Authors and expert reviewers, if time and funding p

ermit, in order to pay special attention to particular points of assessment or areas of major differences.

4.2.4.2 Second Review (by Governments and Experts)

A revised draft should be distributed by the appropriate Working Group /Task Force Bureau Co

- chai rs or through the IPCC Secretariat to governments through the designated Government Focal Points, and to all the coordinating lead authors, lead authors and contributing authors and expert reviewers.

Governments should send one integrated set of comments for each Report to the appropriate Working Group /Task Force Bureau Co

- chairs through their Government Focal Points.

Non

- government reviewers should send their further comments to the appropriate Working Group /Task Force
Bureau Co

- Chairs with a copy to th eir appropriate Government Focal Point.

74

### 4.2.5 Preparation of Final Draft Report

Preparation of a final draft Report taking into account government and expert comments for submission to a Session of a Working Group or, in case of a report prepared by th e Task Force on National Greenhouse Gas Inventories, of the Panel for acceptance should be undertaken by Coordinating Lead Authors and Lead Authors in consultation with the Review Editors. If necessary, and timing and funding permitting, a wider meeting wi th principal Contributing Authors and expert and government reviewers is encouraged in order to pay special attention to particular points of assessment or areas of major differences. It is important that Reports describe different (possibly controversial) scientific, technical, and socio

- economic views on a subject, particularly if they are relevant to the policy debate. The final draft should credit all Coordinating Lead Authors, Lead Authors, Contributing Authors, reviewers and Review Editors by name and affiliation (at the end of the Report).

4.3 Approval and Acceptance of Summaries for Policymakers and Adoption of Overview Chapters of Methodology Reports Related to national Greenhouse Gas Inventories

Summary sections of Reports approved by the Worki ng Groups and accepted by the Panel will principally be the Summaries for Policymakers, prepared by the respective Working Groups of their full scientific, technical and socio

- economic assessments, and Summaries for Policymakers of Special Reports prepared by the Working Groups. The Summaries for Policy Makers should be subject to simultaneous review by both experts and governments and to a final line by line approval by a Session of the Working Group. Responsibility for preparing first drafts and revised d rafts of Summaries for Policymakers, lies with the respective Working Group Co

- Chairs. The Summaries for Policymakers should be prepared concurrently with the preparation of the main Reports.

Approval of the Summary for Policymakers at the Session of the Working Group, signifies that it is consistent with the factual material contained in the full scientific, technical and socioeconomic assessment or Special Report accepted by the Working Group. Coordinating lead authors may be asked to provide technical a ssistance in ensuring that consistency has been achieved. These Summaries for Policymakers should be formally and prominently described as:

"A Report of [Working Group X of] the Intergovernmental Panel on Climate Change."



### For a Summary for Policymakers approved by a Working Group to be endorsed as an IPCC Report, it must be accepted at a Session of the Panel. Because the Working Group approval process is open to all governments, Working Group approval of a Summary for Policymakers means that the Panel ca nnot change it. However, it is necessary for the Panel to review the Report at a Session, note any substantial disagreements, (in accordance with Principle 10 of the Principles Governing IPCC Work) and formally accept it.

Overview Chapters of Methodology Reports related to National Greenhouse Gas Inventories will be adopted section by section by the Panel. The Overview Chapters should be subject to simultaneous review by both experts and governments. Responsibility for preparing first drafts and revised dr afts lies with the respective Task Force Bureau Co

- Chairs. The Overview Chapters should be prepared concurrently with the preparation of the main Reports.

4.4 Reports Approved and/or Adopted by the Panel

Reports approved and/or adopted by the Panel wil l be the Synthesis Report of the Assessment Reports and other Reports as decided by the Panel whereby Section 4.3 applies mutatis mutandis

.

4.4.1 The Synthesis Report

The Synthesis Report will synthesise and integrate materials contained within the Asse ssment Reports and Special Reports and should be written in a non

- technical style suitable for policymakers and address a broad

75

### range of policy

- relevant but policy

- neutral questions approved by the Panel. The Synthesis Report is composed of two sections as

follows: (a) a Summary for Policymakers and (b) a longer report. The IPCC Chair will lead a writing team whose composition is agreed by the Bureau, noting the need to aim for a range of views, expertise and geographical representation. An approval and ado ption procedure will allow Sessions of the Panel to approve the SPM line by line and to ensure that the SPM and the longer report of the Synthesis Report are consistent, and the Synthesis Report is consistent with the underlying Assessment Reports and Spec ial Reports from which the information has been synthesised and integrated. This approach will take 5

- 7 working days of a Session of the Panel.

Step 1: The longer report (30

- 50 pages) and the SPM (5

- 10 pages) of the Synthesis Report are prepared by the

wr iting team.

Step 2: The longer report and the SPM of the Synthesis Report undergo simultaneous expert/government review.

Step 3: The longer report and the SPM of the Synthesis Report are then revised by Lead Authors, with the assistance of the Review Edi tors.

Step 4: The revised drafts of the longer report and the SPM of the Synthesis Report are submitted to Governments and participating organisations eight weeks before the Session of the Panel.

Step 5: The longer report and the SPM of the Synthesis Rep ort are both tabled for discussion in the Session of the Panel:

• The Session of the Panel will first provisionally approve the SPM line by line.

• The Session of the Panel will review and adopt the longer report of the Synthesis Report, section by section, i.e. roughly one page or less at a time. The review and adoption process for the longer report of the Synthesis Report should be accomplished in the following manner:

- When changes in the longer report of the Synthesis Report are required either to conform it to the SPM or to ensure consistency with the underlying Assessment Reports, the Panel and authors will note where changes are required in the longer report of the Synthesis Report to ensure consistency in tone and content. The authors of the longer rep ort of the Synthesis Report will then make changes in the longer report of the Synthesis Report. Those Bureau members who are not authors will act as Review Editors to ensure that these documents are consistent and follow the directions of the Session of t

he Panel

- The longer report of the Synthesis Report is then brought back to the Session of the Panel for the review and adoption of the revised sections, section by section. If inconsistencies are still identified by the Panel, the longer report of the Syn thesis Report is further refined by the Authors with the Assistance of the Review Editors for review and adoption by the Panel. This process is conducted section by section, not line by line.

• The final text of the longer report of the Synthesis Report wil l be adopted and the SPM approved by the Session of the Panel.

The Report consisting of the longer report and the SPM of the Synthesis Report is an IPCC Report and should be formally and prominently described as:

"A Report of the Intergovernmental Pane l on Climate Change."

76

### 5. TECHNICAL PAPERS

IPCC Technical Papers are prepared on topics for which an objective, international scientific/technical perspective is deemed essential. They:

a. are based on the material already in the IPCC Assessment Reports, Special Reports or Methodology Reports ; b. are initiated: (i) in response to a formal request from the Conference of the Parties to the UN Framework Convention on Climate Change or its subsidiary bodies and agreed by the IPCC Bureau; or (ii) as decided by the Panel;

c. are prepared by a team of Lead Authors, including a Coordinating Lead Author, selected by the Working Group /Task Force Bureaux in accordance with the provisions of Sections 4.2.1 and 4.2.2 for the selection of Lead Authors and Coordinating Lead Aut hors. d. are submitted in draft form for simultaneous expert and government review with circulation to expert reviewers and Government Focal Points in accordance with Section 4.2.4.1 at least four weeks before the comments are due;

e. are revised by the Lead Au thors based upon the comments received in the step above; f. are submitted for final government review at least four weeks before the comments are due;

g. are finalised by the Lead Authors, in consultation with the IPCC Bureau which functions in the role of an
E ditorial Board, based on the comments received; and, h. if necessary, as determined by the IPCC Bureau, would include in a footnote differing views, based on comments made during final government review, not otherwise adequately reflected in the paper.

The following Guidelines should be used in interpreting requirement (a) above: The scientific, technical and socio

- economic information in Technical Papers must be derived from:

(a) The text of IPCC Assessment Reports and Special Reports and the portions of mater ial in cited studies that were relied upon in these Reports.

(b) Relevant models with their assumptions, and scenarios based on socio

- economic assumptions, as they were used to provide information in those IPCC Reports, as well as emission profiles for sensit ivity studies, if the basis of their construction and use is fully explained in the Technical Paper.

The Technical Papers must reflect the balance and objectivity of those Reports and support and/or explain the conclusions contained in those Reports.



### In formation in the Technical Papers should be referenced as far as possible to the subsection of the relevant IPCC Reports and related material.

Such Technical Papers are then made available to the Conference of the Parties or its subsidiary bodies, in

resp onse to its request, and thereafter publicly. If initiated by the Panel, Technical Papers are made available publicly. In either case, IPCC Technical Papers prominently should state in the beginning:

"This is a Technical Paper of the Intergovernmental Pa nel on Climate Change prepared in response to a [ request from [the Conference of the Parties to]/[a subsidiary body of] the United Nations Framework Convention on Climate Change ] / [ decision of the Panel ] . The material herein has undergone expert and
gover nment review but has not been considered by the Panel for formal acceptance or approval."

6. IPCC SUPPORTING MATERIAL

Supporting material consists of (i) published reports and proceedings from workshops and expert meetings within the scope of the IPCC w ork programme that have IPCC recognition, and (ii) material
, including databases and software , commissioned by Working Groups, or by the Bureau of the Task Force on National Greenhouse Gas Inventories in support of the assessment or methodology development

, process which IPCC decides should

77

## Appendix B USGCRP/CCSP Procedures and Responsibilities

1) Guidelines for Producing CCSP Synthesis and Assessment Products

2) Clarification of Review and Clearance Process for CCSP Synthesis and Assessment Products

3) Guidance to Agency Leads Regard ing the Preparation of CCSP Synthesis and Assessment Products

88 89

96

105

### Climate Change Science Program

Guidelines for Producing CCSP Synthesis and Assessment Products

The CCSP Synthesis and Assessment Products

The U.S. Climate Change Science Program (CCSP) is producing synthesis and assessment products to support informed discussion and decision making regarding climate variability and change by policy make rs, resource managers , stakeholders, the media, and the general public. The CCSP par ticipating agencies are coordinating their work to produce these reports, which will integrate research results focused on identified science issues and related questions frequently raised by decision makers. These reports will provide current evaluations of the identi fied science foundation that can be used for informing public debate, policy development, and operational decisions, and for defining and setting the future direction and prioriti es of the program. The CCSP products will be considered Federal government disseminati ons, thus they must be prepared in conformance with the provisions of the Data Quality Act (Section 515 of the Treasury and General Government Appropriations Act of 2001). Any agency sponsoring or contributing to the development of a product must certify that the agency’s contribution satisfies its Information Quality Guidelines.

Purpose of the Guidelines

The CCSP Strategic Plan sets forth general principles for its approach to preparing synthesis and assessment products:

• Analyses structured around specific questions

• Early and continuing involv ement of stakeholders

• Explicit treatment of uncertainties

• Transparent public review of analysis questions, methods, and draft results

• Adoption of a “lessons learned” approach, building on the ongoing CCSP analyses.

The purpose of this document is to present guid elines that address the three steps in the process of preparing the synthesis and a ssessment products: developing the prospectus, drafting and revising the document, and fina l approval and publica tion of each product. The guidelines set forth the roles of partic ipants and the steps in the process. The guidelines are intended to ensure that:

• Independent scientific judg ment serves as the guidi ng force in preparing the products so they are credible

• Scientists, users, and other stakeholde rs jointly determine the scope of the products so the topics covered are well defined and the information provided is relevant to the needs expressed

• The process of preparing the products is open at every step so the products have legitimacy (i.e., are perceived to have been prepared fairly).

89

### Participants and Their Roles

CCSP Interagency Committee

CCSP was established by the President in 2002 and integrates the U.S. Global Change Research Program and the Climate Change Research Initiative. The CCSP Interagency Committee provides executive direction for the Program, as described in Chapter 16, “Program Management and Review,” of th e CCSP Strategic Pla n. CCSP’s Interagency Committee is chaired by the CC SP Director (a Department of Commerce appointee) and includes representatives of 13 participating departments/agencies that have mission or funding responsibilities in clim ate and global change research , together with liaisons from the Executive Office of the President. 1 

### Membership on the CCSP Interagency Committee is joint with the Subcommittee on Global Change Research (SGCR) of the Committee on Environment and Natural Resources (CENR) of the President’s National Science and Technology Council (NSTC). The CCSP Interagency Committee has overall responsibility for direction of the program , including compliance with the requirements of the Global Change Research Act of 1990. W ith respect to the synthesis and assessment products, the CCSP Interagency Committee provides oversight for the process of preparing the products as desc ribed in these guidelines.

Lead Agency(ies)/Department(s)

One or more designated CCSP agency(ies) or department(s) will take the lead in producing each product. The lead agency(ies) will be responsib le for developing an open and transparent process for soli citing user input, author nom ination and selection, expert peer review and public commen t, and production/release of the products, as described in these guidelines. To ensure that the products incorporate as much expertise as possible, the lead agency(ies) will be open to the particip ation of other individu als or entities with relevant expertise and information. The en tities can include ot her government units (Federal or non-Federal), Interagency Work ing Groups of the CCSP or other Federal programs, international organizations and government units, non-governmental organizations (NGOs), and other groups.

1 Participating departments and agencies include: Department of Agriculture (USDA), Department of

Commerce / National Oceanic and Atmospheric Administration (DOC/NOAA), Department of Defense (DOD), Department of Energy (DOE), Department of Health and Human Services (HHS), Department of the Interior / U.S. Geological Survey (DOI/USGS), Department of State (DOS), Department of Transportation (DOT), Agency for International Development (USAID), Environmental Protection Agency (EPA), National Aeronautics and Space Ad ministration (NAS A), National Science Foundation (NSF), and Smithsonian Institution (SI). In additio n, the Executive Office of the President and other related programs have designated liaisons who participate on the CCSP Interagency Committee: Office of Science and Technology Policy (OSTP), Counc il on Environmental Quality (CEQ), Office of Management and Budget (OMB), Climate Change Technology Program (CCTP), and Office of the Federal Coordinator for Meteorology (OFCM).

90

### Lead and Contributing Authors

Lead and contributing authors of the synthesi s and assessment products are scientists or individuals with recognized technical expertise appropria te to a product. Lead and contributing authors may be citizens of any c ountry and be drawn from within or outside the Federal government (e.g., universities or othe r public or private sector organizations). These individuals shall be acknowledged e xperts, known through their publication record and relevant accomplishments and contributions to their field. Lead authors are responsible for the content of the synthesis and assessment products that are submitted to the CCSP Interagency Committee for review.

Interagency Working Groups

The CCSP Interagency Committee c oordinates implementation of its activities in support of the Strategic Plan through Interagency Working Groups (IWGs) of program specialists of its participating departments and agenci es, as described in Chapter 16, “Program Management and Review,” of the CCSP Strategic Plan. IWGs will contribute significantly to the preparation of the synthe sis and assessment products because of their expertise in areas related to the products. IWGs may serve as a means for the lead agency(ies) to coordinate preparation of th e products with supporti ng agencies. They may contribute to planning/preparing the pr ospectuses, scoping, drafting, reviewing, publishing, or disseminating the final product.

Expert Reviewers

Expert reviewers are scientists or individuals with other spec ial expertise appropriate to a product. The expert reviewers will be selected by the lead agency(ies)/departments. As is the case for lead and contributing authors, re viewers may be citizens of any country and be drawn from within or outsi de the Federal government (e.g., universities or other public or private sector organizations). These individuals shall be known through their publications and other forms of recognition of their expertise. Expert reviewers will focus on the scientific/technical conten t of the draft. Employees of the lead agency(ies), lead authors, and other contributors to the product may not serve as expert reviewers for that product. The expert reviewers will be desi gnated through a process described in the prospectus.

Stakeholders

Stakeholders are defined as they are in Ch apter 11 of the CCSP Strategic Plan—that is, “Stakeholders are individuals or groups whose interests (financial, cultural, value-based, or other) are affected by climate variability, climate change, or opti ons for adapting to or mitigating these phenomena.” 2 

### Stakeholders participate du ring the scoping process by providing information that help s define the audience and poten tial uses of a product. In addition, stakeholders provide comments on th e prospectus, and on the product during the

2 See Box 11-1 (“Working Definitions”), page 112 of the CCSP Strategic Plan.

91

### public comment period. These comments are expected to focus on how well the product serves its intended purpose or use.

National Research Council

The National Academy of Sciences / Nati onal Research Council (NRC) will provide advice on an as-needed basis to the lead agency(i es). In the event that issues are identified that require further clarificat ion, the NRC may be asked to provide additional scientific analyses to help bound the uncertainty associated with these issues.

National Science and Technology Council

The National Science and Technology Council (NSTC) will be responsible for final review and approval of the synthesis and assessment products. Products not cleared by NSTC cannot be released as disseminations of the Federal government. Consistent with NSTC procedures, approvals will require wr itten concurrence from all members of the NSTC’s Committee on Environment and Na tural Resources (CENR). All comments generated through the NSTC review will be addressed by the CCSP Interagency Committee. The CENR membership includes seni or officials representing the Executive Office of the President and the 15 federal agen cies with significant responsibilities for environment and natural resources programs.

Steps of the Process

Planning the Process and Preparing a Prospectus

1. The lead agency(ies) solicit input from user s and other stakeholde rs, plan preparation of the product, and summarize the proposed process in a draft prospectus. The draft prospectus will address the topics listed in the subsequent section of this document. 2. The CCSP Interagency Committee reviews and approves the draft prospectus for public comment. 3. Expert reviewers and stak eholders review the draft prospectus. The prospectus comment period will last at least 30 days . The draft prospectus comment period will be announced in a Federal Register Noti ce (FRN) and posted on the CCSP web site. 4. The lead agency(ies) revise the draft pros pectus and finalize author recommendations, taking into consideration the comments received. 5. The CCSP Interagency Committee approves the revised prospectus and the lead agency(ies) notify the lead authors. 6. The CCSP Office posts the draft prospectus comments and the final prospectus on the CCSP web site.

Additional Stakeholder Interactions, if Needed

7. Lead authors may solicit additional input fr om users and other stakeholders to assist in the development of the product. The proce ss for soliciting this additional input will be open and described in the prospectus . Approaches include workshops, user

92

### surveys, telephone and email conferences, and other mechanisms. The processes used will reflect the expected end use of the pr oduct. The CCSP Strategic Plan identifies three end uses for CCSP synthesis and assessment products: 1) informing the evolution of the research agenda; 2) s upporting adaptive management and planning; and 3) supporting policy formulation. The pr oducts with end uses primarily oriented toward the second and third categories are expected to require significant additional input from users to develop a clear unders tanding of information needs, timing of decisions, consideration of how uncertainty affects decision making, and other issues. The results from additional stakeholder in teractions will be pub licly available in summary or more extensive forms th rough publication on the CCSP web site.

Drafting/Reviewing the Products

8. Lead authors prepare the first draft, in cluding a technical sec tion and a summary for interested non-specialists. 9. The lead agency(ies) organize and facilitate an expert peer review of the first draft according to process described in the prosp ectus. The expert peer review will precede the public comment period to ensure that the products are sh aped by scientific considerations. The expert peer review process may range from that used in a scientific journal to a formal review panel convened by the lead agency or recognized external groups such as the National Resear ch Council. Participants must be qualified scientific/technical experts, as demonstrated by their reco rd of scholarly publication and other accomplishments. Employees of th e lead agency(ies), lead authors, and other contributors to the product may not serv e as expert reviewer s for that product. The prospectus will describe the process for selecting expert reviewers and the expected dates of the review. If the expert pe er review is open to all qualified experts, notice will be disseminated on the CCSP web site and through relevant scientific publications, web sites, and other means. All comments submitted during the expert peer review will be publicly available w ithout attribution to the reviewer unless reviewers agree in advance to posting with specific attribution. 10. Lead authors prepare the second draft of the product, taking into consideration the expert peer review comments. The scientific judgment of the lead authors will determine responses to the comments. Th e authors will acknowledge significant contributions made by expert reviewers, as applicable. 11. The lead agency(ies) post the second draft of the produc t for public comment for not less than 45 days. Any stakeholders (plus expe rts who participated in the expert peer review process) may particip ate in the public comment pe riod for the second draft. This includes governmental and non-govern mental entities. The prospectus will include the expected date s of the public comment pe riod. Notice of the public comment period will be disseminated on the CCSP web site, in the Federal Register, and through other publications, web sites, and means as appropriate to the product, to encourage wide public participation in th e review. All comments will be publicly available. 12. The lead authors will prepare a third draft of the product, taking into consideration the comments submitted during the public commen t period. The scientific judgment of the lead authors will determine responses to the comments.

9 3

### Approving, Producing, and Releasing the Products

13. Lead agency(ies) submit the third draft of the product and a compilation of comments received to the CCSP Interagency Committee. 14. If the CCSP Interagency Committee review determines that no further action is needed and that the product has been prep ared in conformance with these guidelines and the Data Quality Act (including ensuring objectivity, utility, and integrity as defined in 67 FR 8452), they will submit the product to NSTC for approval. If the CCSP Interagency Committee determines that further revision is necessary, their comments will be sent to the lead agency(i es) for consideration and resolution by lead authors. 15. If needed, NRC can be asked to provid e additional scientific analysis to bound scientific uncertainty associ ated with specific issues. 16. Once the CCSP Interagency Committee has determined that the synthesis and assessment report has been prepared in c onformance with these guidelines and the Data Quality Act, the Committee will submit it to NSTC for final review and approval. Approval will require the concurre nce of all members of the Committee on Environment and Natural Resources. Commen ts generated during the NSTC review will be addressed by the CCSP Interagency Committee. 17. Once NSTC approval has been obtained and the product is finalized, the lead agency(ies) will produce and release the completed product using a standard format for all CCSP synthesis and assessment products. The final product and the comments received during the expert review and the public comment pe riod will be posted, without attribution (unless specific reviewers agree to attribution), on the CCSP web site. 18. The product will be widely disseminate d through the CCSP web site and other mechanisms.

Contents of the Prospectus

The proposed process for preparing each CC SP synthesis and assessment product will be summarized in a prospectus that will be publicly available. The prospectus for each product will typically be 5-10 pages in lengt h (plus appendices with references and biographical information for proposed lead authors) and will address the following points:

• Overview: description of topic, audience, intended use, questions to be addressed,

etc.

• Contact information: email and telephone for responsible individuals at the lead and supporting agencies

• Lead authors: required expe rtise of lead authors and biographical information for proposed lead authors

• Stakeholder interactions: process alrea dy used to solicit input from users and other stakeholders, or proposed plans for doing so, including information for those interested in participating in this process

• Drafting: materials to be us ed in preparing the product

94

### • Review: the processes through which the pr oduct will receive expert peer review and public comment, including the process for selecting expert reviewers and the scheduled dates for the expert peer review and public comment periods

• Related activities: description of how preparation of the product will be coordinated with related activities, in cluding other national or international assessment processes (e.g., the Intergovernmental Panel on Climate Change)

• Communications: proposed method of pub lication and disse mination of the product

• Proposed timeline.

Materials to be Used

Authors will use the published, peer-review ed scientific literature in drafting the products. In the rare case that any materials used in prep aring a product ar e not already published in the peer-reviewed literature, the lead agency(ies) must get approval from the CCSP Interagency Committee and these material s must be made available by the lead agency(ies) and/or CCSP Office. The use of any such non-peer-reviewed materials may be questioned by reviewers during the expert review or public comment period. Authors should seek to publish any materials used in preparing drafts of the products.

Characteristics of the Products

The products will identify disparate views that have significant scie ntific or technical support. They will also provide confidence levels for key findings, if this is appropriate to the product.

95

### MEMORANDUM

TO: CENR and CCSP Principals (see Appendix, which lists these individuals)

FROM: William Brennan, Acting Director, U.S. Climate Change Science Program

RE: Clarification of review and clea rance process for CCSP Synthesis and Assessment Products

Based on “lessons learned” from moving th e first few CCSP synthesis and assessment products through the final review and clea rance stages, this memorandum provides clarifications to the CCSP “Guidelines for Producing Synthesis and Assessment Products” (< http://www.climatescience.gov/Li brary/sap/sap-guidelines.htm >).

ROLE OF THE PRODUCT AUTHORS

The lead and contributing authors are re sponsible for drafting the product. The Guidelines make a clear distinction among the respective roles of authors, expert reviewers, and stakeholders. Contributors may not serve as expert reviewers. All comments pertaining to the content of the produc t that arise at any stage will be referred to the lead author(s). These stages include the expert re view, public comment period, as well as the final clearance stages by the lead agency, CCSP, and NSTC. Any changes that are proposed for the product’s contents will be reviewed by the lead author(s) to ensure that the scientific and technical intent is ma intained. Therefore, the authors must remain available to address comments that arise th roughout the process, including the final CCSP and CENR clearance stage.

It is important that the author s adhere closely to the terms of the product described in the prospectus. Consistent with the princi ples underpinning National Research Council studies: “The rationale for any findings, conclusions, and recommendations should be fully explained in the report. This explanati on might include references to the literature, analysis of data, or a descrip tion of the pros and cons of the range of alternatives and the reasons for preferring a particular opti on. Failure to document conclusions and recommendations adequately is the most common shortcoming of draft reports. Recommendations calling for organizational changes, adoption of new policies or positions, or budgetary increases within government agencies should be avoided...” 1



### ROLE OF THE LEAD AGENCY

Product Review and Clearance

The CCSP “Guidelines for Producing Synthesis and Assessment Products” indicate that following the expert review and public comment stages, the “lead agency(ies) submit the

1 From Guidelines for the Review of National Research Council Reports , 2003

.



### 96

### third draft of the product and a comp ilation of comments received to the CCSP Interagency Committee .” Three clarifications are made below with respect to this

statement.

First, although there may be multiple contributing agencies to a given synthesis and assessment product, a single lead agency must take responsibility for ensuring compliance with the Information Quality Act (IQA), and if applicable, the Federal Advisory Committee Act (FACA).

Second, when the lead agency transmits th e product for final clearance, they should include a memorandum briefly indicating that “the product was prepared in compliance with CCSP’s Guidelines for Producing Synthesis and Assessment Products, the Information Quality Act (Section 515) and [LEAD AGENCY’S] corresponding IQA guidelines; and the Federal Advisory Committee Act [when applicable].” This transmittal should include the authors’ responses to the peer reviewer comments, as required by OMB’s Information Quality Bulletin for Peer Review, as well as descriptions of how the authors addressed the public comm ents and lead agency’s review comments.

Third, to help expedite the clearance pr ocess, CCSP and NSTC clearance steps are conducted simultaneously. The lead agency wi ll therefore simultaneously transmit the necessary materials to both. These materials include: the third draf t of the product; the expert review comments and the author’s re sponses to them; the public comments and the author’s responses to them; and the aforementioned transmittal memorandum.

To further expedite the process, the lead ag ency is encouraged to submit comments it has concerning the report’s conten ts during the public comment stage. The lead agency should not convene an expert review in additi on to the one conducted as part of the report production process.

The lead agency is responsible for informing the authors what their role is in each stage of the product development, review, public co mment, and clearance process, and helping to ensure that the authors remain engage d through the final clearance process.

The lead agency will inform the authors of th e steps that are required to comply with the lead agency’s Information Quality Act guidelines as well as the Information Quality Bulletin for Peer Review, and ensure that the authors follow those steps.

Use of FACA (The Federal Advisory Committee Act)

The determination of whether a FACA Committe e is needed rests with the lead agency. When a FACA Committee is required, the lead agency is responsible for establishing the FACA charter and the process for appointing FACA Committee members. Note that there are other statutory requi rements applicable to FACA Committees, such as those governing Federal Register announcements when a FACA Committee meets, and each lead agency is expected to comply with a ny relevant FACA guidelines. It is also important to note that the role of a FACA Committee is to provide advice to the lead



### 97

### agency on the SAP; the SAP is not a produc t of the FACA Committee, nor is the SAP that is released by the Lead Agency s ubject to approval by a FACA Committee. However, any requests for changes to the produc t’s contents that ar ise in the clearance stage will be referred back to the lead author (s). Any changes that are proposed for the product’s contents will be review ed by the lead author(s) to en sure that the scientific and technical intent is maintained. The report is prepared by the lead agen cy and is ultimately a product of CCSP, a component of the federal government. Therefore, it is essential that agencies authoring SAPs who choose to invoke a FACA process carefully review the FACA Committee advice and take whatever steps are necessary to stand behind the process used to prepare the report and the conclusions therein. In ot her words, the lead agency must "own" the report.

Communications

The lead agency is responsible for devel oping and executing a communications plan for each product. Particular attention should be give n to the rollout plan, which will describe

communication plan be developed early in the product produc tion cycle. When applicable, it should describe the approach es for obtaining stake holder input during the production of the product, including during th e public comment period. It should also describe how and to whom the product wi ll be communicated following its completion. The plan should allow for flexibility in pl anning communications-rel ated activities given the possibility of changing timetables. The plan should be developed in consultation with the lead agency’s public affairs and communi cations offices, and should draw from the “best practices” document maintained by CC SP’s Communications In teragency Working Group. The plan should be communicated to th e Executive Office of the President (EOP) and the CCSP Office.

ROLE OF CCSP AND NSTC

The Global Change Research Act of 1990 re quires "the Council [NSTC], through the Committee [CENR]" to deliver scientific assessment products to the President and Congress. Accordingly, the NSTC and CENR must review the a ssessment products prior to their release. Pursuant to the Act, the CCSP clearance process will occur via the NSTC, and will be conducted through the CENR.

Because CCSP and NSTC clearance will be conducted simultaneously, and because some agencies have members on both the CCSP Principals Interagency Committee and the CENR, those agencies should coordinate thei r clearances so that each agency provides only one clearance.

The CCSP/CENR clearance will rely on the lead agency's certification regarding compliance with CCSP's "Guidelines for Produc ing Synthesis and Assessment Products," FACA, and IQA (including the Bulletin for Peer Review). As such, an additional expert review is not expected. The focus of the CCSP/CENR clearance is to ensure that the the strategy used for the releas e of the report to the public. It is recommended that the



### 98

### policy officials with legal responsibility fo r the documents have reviewed and approved them, with particular consider ation of issues such as:

 The product does not misstate any policies or pos itions of the United States or any of its federal agencies. (The synthesis a nd assessment products are intended to be policy-relevant, not policy-prescriptive.)

 The product clearly communicates its main message to its intended audiences and includes sufficient context for members of the lay public who are interested in the science to understand th e product’s relevance.

 The expert review and public comments we re adequately addre ssed by the authors.

Any requests for changes to the product’s conten ts that arise in the clearance stage will be referred back to the lead author(s). Any changes that are proposed for the product’s contents will be reviewed by th e lead author(s) to ensure that the scientific and technical intent is maintained.

PUBLIC ACCESS TO DOCUMENTS

The following is the minimum set of documents that are posted on the web page for each of the synthesis and assessment products ( http://www.climatescience.gov/ Library/sap/sap-summary.php )

• invitation for public comment on draft prospectus

• draft prospectus

• public comments on draft prospectus

• final prospectus

• peer review plan (if not part of the prospectus)

• expert review comments on first draft report and authors’ responses to them

• invitation for public comment on second draft report

• second draft report

• public comments on second draft report and authors’ responses to them

• third draft report

• final report

Additional information is often posted for each product.

EXECUTIVE SUMMARY FOR THE PRODUCTS

Each product should include an executive summary. An effective executive summary contains enough information for the reader to become acquainted with the full document without actually reading it. It should briefly summarize: the key issue(s) being addressed by the product; the background and context of the issue(s); and the major conclusions. Recognizing that the 21 synthesis and assessment products are scientific in nature, the executive summary should be written for a non-expert (e.g., to the level of an informed



### 99

### high school graduate) and minimize the use of technical terminology and graphs and figures. It is recommended that the executiv e summary be no more than eight pages in length.

It is up to the discretion of the author teams whether or not to also produce an abstract that succinctly distills the essence an d key take-home messages of the report.

Both the executive summary and the abstract (if included) are independent elements and should undergo the same review and clea rance as required for the full report.

10 

### 0

### APPENDIX

CENR ROSTER

George Gray U.S. Environmental Protection Agency Headquarters

Conrad Lautenbacher National Oceanic and Atmospheric Administration

Sharon Hays Office of Science and Technology Policy Executive Office of the President

Michael Freilich National Aeronautics and Space Administration

Jonathan Perlin U.S. Department of Veterans Affairs

James Connaughton Council on Environmental Quality

Vacant U.S. Department of Energy

Robert Foster Office of the Secretary of Defense

Mark Myers U.S. Geological Survey Department of the Interior

Leonard Hirsch Smithsonian Institution

Kathryn Jackson River System Operations & Environment Tennessee Valley Authority

Gale Buchanan Research, Education, and Economics U.S. Department of Agriculture

Linda Lawson Safety, Energy and Environment

10 

### 1

### Department of Transportation

Kathie Olsen National Science Foundation

Vacant U.S. Department of Housing & Urban Development

David Schwartz U.S. Department of Health & Human Services

David Anderson The Office of Management and Budget Executive Office of the President

Bruce Davis U.S. Department of Homeland Security

Vacant U.S. Department of Labor

Claudia McMurray Bureau of Oceans and Int’l Envi ronmental & Scientific Affairs U.S. Department of State

Samuel Williamson Federal Coordinator for Meteorology

EOP CC list Chase Hutto Office of the Vice President

Ted Wackler Office of Science and Technology Policy Executive Office of the President

Marty Hall Council on Environmental Quality Executive Office of the President

Other CC list Marta Cehelsky Executive Secretary NSTC Committee on Science and Senior Adviser, Office of the Director National Science Foundation

10 

### 2

### CCSP roster

William Brennan, Acting Director National Oceanic and Atmospheric Administration Department of Commerce

Jack Kaye, Vice Chair HQ/Earth Sun System Division National Aeronautics and Space Administration

Allen Dearry National Toxicology Program National Institute of Environmental Health Sciences National Institutes of Health

Jerry Elwood Climate Change Research Division Department of Energy

Mary Glackin National Oceanic and Atmospheric Administration Department of Commerce

Patricia Gruber Office of Naval Research Department of Defense

William Hohenstein Global Change Program Office U.S. Department of Agriculture

Linda Lawson Safety, Energy and Environment Department of Transportation

Jarvis Moyers Geosciences Directorate National Science Foundation

Mark Myers U.S. Geological Survey Department of the Interior

Patrick Neale Smithsonian Environmental Research Center.

10 

### 3

### Smithsonian Institution

Jacqueline Schafer Bureau for Economic Growth, Agriculture, and Trade U.S. Agency for International Development

Joel Scheraga National Center for Environmental Assessment U.S. Environmental Protection Agency

Harlan Watson Department of State

Executive Office and other Liaisons to the CCSP Principals Committee

Melissa Brandt Office of Management and Budget Executive Office of the President

Stephen Eule Climate Change Technology Program U.S. Department of Energy

Katharine Gebbie National Institute of Standards & Technology

Margaret McCalla Office of the Federal Coordinator for Meteorology

George David Banks Council on Environmental Quality Executive Office of the President

Gene Whitney Office of Science and Technology Policy Executive Office of the President

Staff

Peter Schultz Climate Change Science Program Office

Fabien Laurier Climate Change Science Program Office

10 

### 4Guidance to Agency Leads R egarding the P reparation of CCSP Synthesis and Assessment Products

1. General guidance

General guidance on preparing a Synthesis and Assessment Product ( SAP ) is posted on the CCSP website . A PDF version is available by click ing on the following link : http://www.climatescience.gov/Library/sap/sap

- guidelines.pdf

. CCSP has formed a work group to assist in the preparation of the SAP’s. This work grou p, known as the Synthesis and Assessment Product Advisory Group (SAPAG), has representatives from every agency that is leading in the development of an SAP 1

. The purpose of this communication is to provide additional information and clarification on the production of the SAP’s.

2 . Instructions on the length of an executive summary

Each SAP is to include an abstract and an executive summary. An effective executive

summary c ontains enough information for the readers to become acquainted with the full document without actually reading it. Usually, it contains a statement of the problem, some background information, the context of the study and the major conclusions. Recognizing that the 21 SAP’s are scientific in nature, the executive summary should b e written to the level of a high school graduate and minimize the use of technical jargon and complicated graphs and figures. We are recommending that the executive summary be no more than eight pages in length

. Both the executive summary and the abstrac t are independent elements

. With the possible exception of the conclusion and recommendation, the executive summary is the most important part of a report. As such, it should be the best

- written and most polished piece of the document. This is because ma ny readers may only look at the executive summary when deciding whether or not to read the entire document. Since the executive summary is a condensation, when creating it, you should omit any p reliminaries, details, and illustrative examples. You would i nclude the main ideas, the facts, the necessary background to understand the problem, and the major conclusions. Brevity and conciseness are the keys to a well

- written

summary. Do not take a few sentences from key sections of the document and string them together. Rather, go over the entire document and make notes of the elements you consider important. From your notes, create a rough draft of the summary. Then, polish what you have written until it is smooth and seamless without unnecessary wordiness. Fin ally, ensure that your executive summary is accurate and representative of your full document. It should not be misleading, but it should give readers the same impression as if they had read the entire report. The Executive Summary is to be produced by th e author team and would undergo the same review and clearance procedures required of the overall body of the SAP.

The Communications IWG has volunteered to help you with this process. Please contact the CIWG if you would like their assistance in producing the Executive Summary or providing an independent check on its readability. The Co

- Chairs are Kent Laborde, NOAA (kent.laborde@noaa.gov) and Jason Samenow, EPA ( samenow.jason@epa.gov ), and the CIWG coordinator is Nick Sundt, CCSPO (nsundt@usgcrp.gov).

3. Peer R eview



### 1 05As the agency lead for a Synthesis and Assessment Product you are responsible for

conducting a formal peer review of the report. You must follow your agency’s guidelines for peer review that are consistent with OMB’s Final Information Quality Bulletin for Peer Review ( http://www.whitehouse.gov/omb/inforeg/peer2004/peer_bulletin.pdf ). This Bulletin establishes minimu m standards for peer review and was issued under the Information Quality Act and the OMB’s general authorities to oversee the quality of agency information, analyses, and regulatory actions. All of the 21 SAP’s are determined to be “Highly Influential Sci entific Assessments” and the OMB Bulletin details specific procedures and requirements for conducting peer review including the desire to conduct the peer review as a panel rather than a letter review, the timing of the review, the scope, the selection of reviewers (expertise and balance, conflicts, independence), public participation and disposition of comments. Note that this OMB bulletin calls for a peer review plan for your assigned SAP. These peer review plans may be included in your prospectus, but it must also be posted on your agency’s website, pursuant to the Information Q uality Peer Review Bulletin . Although the CCSP will be posting the prospectuses and final reports, the posting of peer review plans is left up to each agency. If you choose to post an intermediate draft of the SAP on your own agency’s website, please be sure and include the following disclaimer as required by the OMB bulletin:

THIS INFORMATION IS DISTRIBUTED SOLELY FOR THE PURPOSE OF PRE

DISSEMINATION PEER REVIEW UNDER APPLI CABLE INFORMATION QUALITY GUIDELINES. IT HAS NOT BEEN FORMALLY DISSEMINATED BY [List your agency here]

. IT DOES NOT REPRESENT AND SHOULD NOT BE CONSTRUED TO REPRESENT ANY AGENCY DETERMINATION OR POLICY

In these instances, CCSP will not be posting a pre

- dissemination draft of the SAP on the CCSP website. Additional information is attached (see May 19, 2005 memo from Jim Mahoney and Richard Moss).

4. I nformation Q uality P rocedures

As a lead agency for an SAP, you will be responsible for complying wit h the Information Quality Act. Section 515 of Public Law 106

- 554, known as the Data Quality Act, required the Office of Management and Budget to promulgate guidance to agencies ensuring the quality, objectivity, utility, and integrity of information (incl uding statistical information) disseminated by Federal agencies. OMB's government

- wide guidelines, published as interim final on September 28, 2002 (66 F.R. 49718) and finalized on February 22, 2002 (67 F.R. 8452), can be found on OMB’s OIRA's website http://www.whitehouse.gov/omb/inforeg/agency_info_quality_links.html

. To facilitate any requests that might ensue pursuant to this statute

, it is a good idea that you advis e each of your authors to:

a. Retain a copy of any document they cite; b. Identify specific pages in the literature they cite, where appropriate; and c. Use explanatory footnotes whe n the text extends beyond a fact cited in the literature.

Each product team should also be prepared to make available any data it uses in the report’s charts or figures. These data should be publicly available as early as the expert review and public comment phases and should remain available thereafter.



### 1065. FACA (The Fede ral Advisory Committee Act)

The determination of whether a FACA committee is needed rests with the lead agency for the SAP. When a FACA Committee is required, t he lead agency is responsible for establishing the FACA charter and the process for appointing FACA Committee members. Note that there are specific requirements for Federal Register announcements when a FACA Committee meets and each SAP lead agency is to comply with its own agency FACA rules. Originally, CCSP was going to provide guidance on the F ACA process , but we realized that such guidance would possibly subordinate agency procedures that have already been adopted. Thus, it is up the assigned agency to comply with FACA accordingly.

6. Progress Reporting

6.1. Introduction

In an effort to s treamline the communications process among the U.S. Climate Change Science Program (CCSP) Synthesis and Assessment Products (SAP) lead authors, the CCSP Office (CCSPO) has developed a Web

- based system for entering and tracking the status of the various SAP products.

The SAP status tracking system was designed using a commercial Web

- publishing software package called ExpressionEngine. The two main components through which users will use this system are the Control Panel page and the end

- user Status page, bo th of which are password

protected. The Control Panel page is the administrative interface where you will enter SAP status information. Once entered, the information will be available on the SAP Status page: http://www.usgcrp.gov/sap/status/

Note that the software was upgraded in February 2006. These instructions are revised to reflect changes in the program’s Administration Panel.

6.2. Logging on

To access the Control Panel page, go to: http://blogs.usgcrp.gov/system

- 20050124

- admindir/index.php

and enter the username and password you were provided.

There is also a link to the Control Panel page from the main SAP site: http://www.usgcrp.gov/sap/

6.3. Editing SAP Status Entries

The Control Panel page will have a menu bar on the top with the following tabs:

PUBLISH | EDIT | MY ACCOUNT

To edit the informatio n for a particular SAP, click on the EDIT tab. On the EDIT page, select “Synthesis and Assessment Products (SAP) Status” from the dropdown menu at the upper left. Beneath that dropdown box, select “Alphabetical” from the “Order” dropdown menu. Then cli ck



### 107on the Search button. This will produce a complete list of the SAPs. Find the product you wish

to edit and click on the product name.

That will take you to a 3

- tab editing page. The three tabs are:

PUBLISH FORM | DATE | CATEGORIES

Start with the “Publish Form” tab

A. “PUBLISH FORM” TAB

a) Entering and Selecting Information

“Title”

This will already be entered and should NOT be changed in any way, as this will change the hyperlink to the product.

“Full Product Name” “Status Summary” “Immediate Concerns” “Action Required (what and by whom)” “Communications Activities, Stakeholder Interaction” “Product's Public Home Page”

These fields are each set up as text boxes. Simply enter the appropriate information in the fields.

The remaining fields are set

- up in the form of a dropdown menu followed by a text field and relate to the following items:

 FACA Process Initiated

 FACA Charter completed

 Draft Prospectus Completed

 Draft Prospectus Approved by CCSP

 Draft Prospectus Publicly Reviewed

 Prospectus cle ared for publication

 Prospectus published

 First draft completed

 Expert review completed

 Second draft completed

 Second draft publicly reviewed

 Third Draft Completed

 CCSP Review Completed

 Approved by NSTC

 Published on the Web

 Hardcopy published

For each ite m, the dropdown menu indicates whether or not the item has been completed (the options are “Yes” and “No”) and the date on which it was completed. Where noted on the form, please enter the date in the yyyy

- mm

- dd format:



### 1082005

- 12

- 31 (not 31 December 2005; 10 December, 2005; Dec. 31, 2005 or 12/31/05)

Click the blue and white “Quick Save” button in the green box near the upper right portion of the PUBLISH page to save your changes.

b) HTML Formatting and Adding Links

In any of the text fields, you can apply standard HTML formatting to any text you wish, including bold, underline, etc. You can also add hyperlinks to Web sites or e

- mail addresses and even images, should you so desire.

All of these formatting options are controlled via the html toolbar:

T o embed Web link in part of the text, select the text and click the Link button on the html toolbar and enter the corresponding URL. You will automatically be prompted to enter the title for the hyperlink. This is the text in which the link will be embedde d. By default, this will be the text you selected. You will also be prompted to enter a “title attribute,” which is the text that will appear when users hold their mouse pointers over the link. Again, by default, this will be text that you selected. You ca n also add a Web link without first selecting text in which to embed it.

You can also add or embed e

- mail addresses anywhere in text field. The process is similar to that of adding or embedding a Web link. You can either select text (typically a person’s name or e

mail address) in which to embed the link or simply add it from scratch. In either case, start by clicking the Email button. You will be prompted to enter an e

- mail address and a link title. If you select text before clicking the Email button that text will be entered as the default link title. You can also leave the link title field empty if you would like the e

- mail address to be displayed at the link title (note: if you select text but leave the link title field blank, the selected text will be replaced with the e

- mail address).

You can add an image (such as a logo) to an event entry by clicking the Image button. The image must already reside somewhere on the Web and you will be prompted to enter the URL for the image. It is possible to change t he size at which an image is displayed, but to do so requires manually entering the appropriate html code after the image source tag.

You can apply various html tags to text anywhere in the entry. These include bold (<b>), italics (<i>), underline (<u>), (<bq>) and strikethrough (<strike>). To apply any of these tags, first select the text to which you want the tag applied and click the appropriate button on the html toolbar. The html tags will automatically be inserted around the selected text.

Don’t for get to click the blue and white “Quick Save” button in the green box near the upper right portion of the PUBLISH page to save your changes.

B. “DATE” TAB



### 109Go to this tab to enter the date on which this SAP was last updated. As this date is not

automatical ly entered, it is important that you enter the date on which you make any changes to an SAP product’s status information.

You can either enter the date in the text box or click on the “Date Calendar” icon to select the date.

If you are not planning any further changes, click on the “Update” button on the right side of the screen. If you wish to make additional change under one of the other tabs (CATEGORIES or PUBLISH FORM), click “Quick Save” to save your changes before going to the another tab.

“Expi ration Date”

Please ignore this field, as it does not apply to the SAP status information.

C. “CATEGORIES” TAB

As various steps of a product’s review, submission, and release are completed, progress should be indicated by selecting all completed steps l isted in the “Categories” tab. The various steps available are:

Draft Prospectus Publicly Available Final Prospectus Publicly Available Peer review of First Draft Product Completed Public review of Second Draft Product Completed Third Draft Product Submit ted to CCSP Product Submitted to NSTC Final Product Released

If only the first step (Draft Prospectus Publicly Available) has been completed, select only that step. If multiple steps have been completed, hold down the Shift key and select the most recent step. This should ensure that all completed steps are selected.

If you are not planning any further changes, click on the “Update” button on the right side of the screen. If you wish to make additional change under one of the other tabs (DATE or PUBLISH FORM), click “Quick Save” to save your changes before going to the another tab.

6.4. Questions/Help

For general questions relating to the ExpressionEngine blog software, you can try searching the online user guide. Simply click the User Guide link on the top

- right corner of the window to access the guide, which will open in a new window.

If you cannot find the answer to your question in the User Guide or if you have a specific question relating to an event you entered, please contact Nick Sundt at nsundt@usgcrp.gov and he will be happy to assist you.



### 1107. Communications

Communications is one the four overarching pillars of the Climate Change Science Program. Developing a sound, comprehensive communications str ategy for the release of each Synthesis and Assessment Product (SAP) should be a priority for the responsible lead agencies in order to broadly share and promote the findings of these considerable efforts.

This document offers recommendations for developi ng communications plans for the SAPs as well as lessons learned from communications activities that were conducted for SAP 1.1. It is intended for SAP lead authors and their associates.

This is a living document. As more SAPs are publicly released and new knowledge is gained on successful communications strategies, this document will evolve. The CIWG and SAPAG encourage individuals involved in SAP communications activities to provide feedback based on their experiences.

7.1 . Communications Plan Guidi ng Principles

 Communications are responsibility of Lead Agency for each product, and a critical component of each SAP.

 Early on: Determine what needs to be done, by whom and when

o Develop calendar/timeline with specific events and dates

 Identify in prospec tus detailed communications strategies including the identification of key stakeholders

 Engage stakeholders throughout the process by developing contact lists, soliciting feedback on drafts and convening workshops and briefings

 Consult with CIWG and SAPAG, and request guidance/assistance as necessary

 Consider embedding a communications expert in the report writing process and/or an editor

 Keep lead agency public affairs and communications offices informed of product process and progress from the beginning; keep Executive Office of President (EOP) informed through CCSP

 Allow for flexibility in planning communications

- related activities given the possibility of changing timetables

7.2 . Elements of Communications Plan and Recommendations

 Dissemination Mater ials

– produce consistently presented materials in both hard copy and electronic form (for dissemination via web and email); all dissemination materials, other than the press release, should be approved by the author team and peer

- reviewed along with the e ntire report.

o Executive Summary* (see Appendix 1)

 Ideally 2

- 4 page summary (no longer than 8 pages) written at high school reading level, including easy to understand figures and background, explaining the relevance of the report.

 A communication professi onal and/or CIWG representative could help produce this overview.

 Available within the SAP or as a stand

- alone product

o Full report

 Develop list of stakeholders for electronic and hard copy distribution



### 111o Press release

 Talking points and questions and answers

o Prepare iteratively with lead agency’s Office of Public Affairs and in coordination with CCSPO prior to final product release

 Legislative briefing(s)

o Following the expert review and revision stage ( optional )

 Work with CCSP/Lead Agency Congressional office s to prepare list of legislative contacts

 Notify EOP via CCSP and appropriate officials at Lead Agency about the Product and intent to notify Capitol Hill

 Offer optional briefing to Hill and schedule upon request

o Release of final report

 Notify EOP via CCSP and Lead Agency of intent to schedule Hill briefings

 Schedule Hill briefings

 Reporters briefing

o Plan ahead

— give reporters sufficient notice of briefing (about 1 week)

o Provide reporters with pre

- reading: embargoed press release, executive summary and abstr act 1 day in advance of public release

 Constituent briefing

o Plan ahead of time:

 identify speakers

 determine who to invite and how to publicize

 assess venue options and location

o Convene shortly after product release date (within two weeks after release f or maximum effectiveness.)

o Keep it short and simple (should not have more than three speakers, and should not last more than 90 minutes, dividing the time between presentations and open Q&A). Discussion should focus on implications / relevance, rather than detailed technical discussions.

 Evaluation Metrics

o Track and report:

 Who signed up to receive the report and who the report was distributed to (in hard copy and electronically)

 Website downloads and related analytics

 Media coverage

7.3

. Role of CIWG

 May serve as an informal consultant for any aspect of the development of S&A Product communications plans

 Lead agencies’ CIWG members may provide useful assistance in writing/reviewing communications products (press releases, executive summary, etc.), plannin g/convening briefings and workshops, and disseminating products

 Contact CIWG Co

- Chairs Kent Laborde < kent.laborde@noaa.gov > and Jason Samenow

< samenow.jason@epa.gov > for guidance/assistance

*Appendix 1: Executive Summary Guidance (from CIWG Implementation Plan through 2007)



### 112Since many of the constituencies that are within the intended audience for the SAPs are not

specifically scientific, it is essential to provide a plain

- language overview to accompany the technical body of the SAP, or possibly stand alone in certain circumstances. There is great utility in providing a non

- technical explanation of findings contained within the SAP, as described in the CCSP Strategi c Plan Communications Section. The Executive Summary would be produced by the author team and would expect the same expert review and clearance procedures required of the overall body of the SAP. It should be no longer than eight pages, and should be writt en with a

non

- technical reader in mind.

SAP lead author teams will prepare the Executive Summary with assistance from the CIWG, as requested.

_________________________________________

1 SAPAG Membership Michael Slimak, EPA and Chair Fabien Laurier , C CSPO Clare Sierawski, DOT Jerry Elwood, DOE L. Dewayne Cecil, NASA Tom Armstrong, USGS Rick Rosen, NOAA Margaret Walsh, USDA



### 1138. Guidance to SAP Agency Leads on Handling Uncertainty

Synthesis and assessment of scientific findings exists at the interfa ce of discovery and societal need

–

it is an interdisciplinary, collaborative production and communication of knowledge , not just information . A healthy synthesis and assessment process must explicitly address uncertainty. This is necessary to solidify the credibility of the research effort underlying the assessment, to build trust from stakeholders, clients, and users, and ultimately to produce something relevant.

The 21 U.S. Climate Change Science Program (CCSP) Synthesis and Assessment Products (SAPs ) c over observations of past and present climate change, climate drivers, model

- based projections of future climate change, vulnerability and impacts, and decision support. Characterizing, quantifying, and communicating uncertainty in climate change assessmen ts presents challenges for a variety of reasons, including multiple spatial scales from global to local; long timescales (compared to instrumental records) and long time lags between forcings and responses; reliance on linked systems of complex models; non linear interactions between systems; lack of near

- term verification of future predictions; and participation of subject experts from multiple disciplines, each with different norms for treating uncertainty.

Recently, the Intergovernmental Panel on Climate Change (IPCC) and the CCSP have produced a small number of guidance documents on handling uncertainty in climate change assessments. Notably:

Moss, R. and S.H. Schneider, 2000: Uncertainties in the IPCC TAR: Recommendations to lead authors for more consi stent assessment and reporting. In: Guidance Papers on the Cross Cutting Issues of the Third Assessment Report of the IPCC [Pachauri, R., T. Taniguchi, K. Tanaka (eds.)], World Meteorological Organisation, Geneva, Switzerland, 33

- 51.

IPCC, 2004: Workshop on Describing Scientific Uncertainties in Climate Change to Support Analysis of Risk and of Options. Working Group I Technical Support Unit, Boulder, Colorado [Manning, M., M. Petit, D. Easterling, J. Murphy, A. Patwardhan, H.

- H. Rogner, R. Swart and G.
Yo he (eds.)], May 11

- 13, 2004, National University of Ireland, Maynooth, Co. Kildare, Ireland, 146 pp. Available at: http://ipcc

- wg1.ucar.edu/meeting/URW/product/URW_Report_v2.pdf

.
Morgan, M.G., H. Dowlatabadi, M. Henrion, D. Keith, R. Lempert, and T. Wilbanks, 2006: Best practice approaches for characterizing, communicating and incorporating scientific uncertainty in climate decision making. SAP 5.2 [report in draft form].

This memo distills certain key themes, lessons, and recommendations from these reports, drawing most heavily from SAP 5.2, to provide guidance for SAP Agency Leads.

8.1. Types of Uncertainty:

Qualitatively different types of uncertainty need to be consider ed and addressed when presenting scientific findings. Most literature makes a distinction between “statistical” uncertainty and “structural” uncertainty: a number of related terms are also employed, often interchangeably. Statistical uncertainty refers to parameters or observed values that are not known precisely. Structural uncertainty refers to a basic lack of knowledge about processes and relationships that



### 114we wish to capture: i.e., whether or not our conceptual model of the world accurately reflects

rea lity and the extent to which our physical/analytical/numerical models encapsulate this understanding. The difference between the two is sometimes summarized as uncertainty resulting from lack of information versus that resulting from ambiguity or disagreem ent over what is known (or even knowable). Quantifying structural uncertainty is generally much less straightforward and consequently generally receives less attention in the scientific literature, even though it is arguably more important when dealing wit h climate change.

Sources of statistical uncertainty relevant to climate change include lack of data; inadequate or incomplete measurement; variation arising in measurement instruments and methods; systematic error and the subjective judgments needed to e stimate its nature and magnitude; and inherent randomness and variability of natural systems.

Sources of structural uncertainty relevant to climate change include inadequate or incomplete measurements and data which prevent the elimination of plausible al ternative hypotheses; systematic errors which drive misleading interpretations of underlying mechanisms; inadequate imagination and inventiveness in suggesting model structures; disagreement among different experts about interpreting available evidence; in complete or competing conceptual frameworks; and ambiguous system boundaries or definitions.

8.2. Communicating Uncertainty in SAPs:

Critical information about the nature and sources of uncertainty are easily buried in imprecise and opaque language.

It is important to recognize the basic differences between descriptions of uncertainty in terms of “likelihood” or in terms of “level of confidence” of the science. Likelihood is the chance of a defined occurrence or outcome, expressed in a probabilistic way. Level of confidence refers to the degree of belief in the scientific community that available understanding, models, and analyses are accurate, expressed by the degree of consensus in the available evidence and its interpretation. Both are important when dealing with climate change and both must be communicated.

When expressing likelihood, there are many words used to describe different degrees of uncertainty: “probable,” “possible,” “almost impossible,” “likely,” “unlikely,” etc. Such qualitative languag e is inadequate because the same words can mean very different things to different people, and the same words can mean very different things to the same person in different contexts. The above

- cited guidance documents strongly urge assigning subjective

num erical probabilities to such qualifiers and applying them consistently.

When dealing with the level of confidence in our scientific judgments about climate change and its impacts, it is important to consider two attributes: the amount of evidence availabl e to support the judgment being made and the degree of consensus within the scientific community about that judgment. The state of knowledge underlying any judgment can then be sorted into, for example, four categories (as in Fig. 1.1 in SAP 5.2): well est ablished; established but incomplete; competing explanations; and speculative.

8.3. Key Issues Related to Scenarios and Modeling

Most of the SAPs, particularly those falling under Goals 2, 3, and 4, draw in some way from future scenarios of socioeconomic changes, and the model projections of climate changes and impacts driven by this forcing. In this context, a few additional reminders:



### 115 Models contain a huge variety of explicit and implicit expert judgments, each to some degree subjective.

 Uncertainty te nds to increase going from global to regional to local scales. This is because of increased inherent climate system variability at smaller scales and resolution limitations of our observational data and modeling tools.

 “Downscaling” of coarse

- resolution cl imate information to finer scales, for example using statistical methods or high

- resolution regional climate models, introduces additional uncertainties.

 Uncertainty tends to increase going from a climate system change to an ecosystem or human impact. This is because of the resolution issues mentioned above, the nonlinear dependence of many impacts on changes in climate system variables, and the relatively less well developed models of these impacts.

 Techniques such as ensemble prediction, Monte Carlo analy sis, etc. can add probabilities to model

- based findings. These formal probabilities will only capture a portion of the total uncertainty of the overall problem.

 Increased “convergence” (across simulations from different models) and reduced “bias” (in the a bility of models to reproduce observed past and present conditions), along with similar metrics, are not guarantors of improved simulations of future conditions.

 Unpredictable behaviors emerge when system complexity increases and multiple feedbacks interac t (for example, when linking together multiple models or going to higher resolution). It is particularly important not to underestimate uncertainty when assessing the possibility of threshold effects and low

- probability but high

- consequence events from cur rent scientific understanding and modeling tools.

 It is much more difficult to handle socioeconomic and technological factors underlying emissions scenarios, or, similarly, mitigative or adaptive capacity, deterministically, as compared to predictions abou t the future physical state of the climate system. Therefore, quantifying uncertainty associated with any particular socioeconomic scenario or assumption about future human responses is less meaningful than defining how a set of scenarios bounds the ranges of a few key variables and being explicit about which factors are and are not considered, and/or only considered exogenously.

Bear in mind that our understanding of complex problems such as climate change may grow richer without any reduction in perceive d levels of uncertainty. Reduced uncertainty in some areas may be offset by newly

- revealed processes or unanticipated complications. Therefore, efforts to identify and understand the full range of uncertainties is a role for the assessment process that is just as critical as identifying where research has narrowed the ranges of specific uncertainties.

8.4. Bottom Line Guidance:

CCSP is not providing prescriptive guidance for handling uncertainty in the production of the 21 SAPs. Rather, in the context of the topic of their SAP, each lead Agency is expected to address uncertainty explicitly by drawing on the guidance summarized here and the above

- cited references. Agency leads are advised to

(1) Instruct authors to incorporate explicit discussions of uncert ainty throughout their report. There should be a clear assignment of responsibility for this within the author team. (2) Instruct authors to familiarize themselves with the material contained in the above

- cited guidance documents.



### 116(3) Instruct authors to create a checklist of sources of uncertainty, organized under the two fundamental types, statistical and structural, and to apply this checklist when developing the findings of their report. For each major finding, identify the most important sources that are likel y to affect the conclusions. (4) Instruct authors to use uncertainty qualifiers precisely and consistently, with associated subjective probabilities, when expressing likelihood. (5) Instruct authors to express the level of confidence in the current scientific unde rstanding of an issue by being transparent about the amount of evidence available and the degree of consensus in the scientific community surrounding that issue. In the face of wide disagreement in the literature, presenting the range of views represented is often more meaningful than attempting to enforce an artificial consensus.

Prepared by Chris Weaver, EPA’s Global Change Research Program Washington, DC Oct 2, 2006



### 117

## Appendix C NRC Report Development Procedures

1) The National Academies: Our Study Process

– Ensuring Independent, Objective Advice

2) The National Academies: Getting to Know the Committee Process

3) Report Review Committee Guidelines for Review: C onsensus Reports

118 119

121

141sub mission of information by outside parties; 3) reviews of the scientific literature, and 4) the investigations of the committee members and staff. In all cases, efforts are made to solicit input from individuals who have been directly involved in, or who have special knowledge of, the problem under consideration.

In accordance with federal law and with few exceptions, information-gathering meetings of the committee are opento the public, and any written materials provided to the committee by individuals who are not officials, agents, or employees of the National Academies are maintained in a public access file that is available for examination.

The committee deliberates in meetings closed to the public in order to develop draft findings and recommendations free from outside influences. The public is provided with brief summaries of these meetings that include the list of committee members present. All analyses and drafts of the

r eport remain confidential.



### STAGE 4. Report Review

As a final check on the quality and objectivity of the study, all National Academies reports—whether products of studies, summaries of workshop proceedings, or otherdocuments—must undergo a rigorous, independent external review by experts whose comments are provided anonymously to the committee members. The National Academies recruit independent experts with a range of views and perspectives to review and comment on the draft

r eport prepared by the committee.

The review process is structured to ensure that each report addresses its approved study charge and does not go beyond it, that the findings are supported by the scientific evidenceand arguments presented, that the exposition and organization are effective, and that the report is impartial and objective.

Each committee must respond to, but need not agree with, reviewer comments in a detailed “response to review” that is examined by one or two independent report review “monitors” responsible for ensuring that the report review
criteria have been satisfied. After all committee members and appropriate National Academies officials have signed off on the final report, it is transmitted to the sponsor of the study and is released to the public. Sponsors are not given an opportunity to suggest changes in reports. The names and affiliations of the report reviewers are made public when the report is released. 

### STAGE 3. Committee Meetings, Information Gathering, Deliberations, and Drafting the Report

Study committees typically gather information through: 1) meetings that are open to the public and that are announced in advance through the National Academies Web site; 2) the 

## HOW THE PUBLIC CAN FOLLOW AND PROVIDE INPUT TO STUDIES

The Current Projects System was established

with a link from the National Academies homepage, www.national-academies.org , to make it easy for members of the general public with interest in the subject to follow the progress of a study and submit comments. The system offers separate views by subject and by project title.

Reports of the National Academies are available from the National Academies Press, 500 Fifth Street, NW, Washington, DC 20001 1-800-624-6242 • www.nap.edu.interest, no individual can be appointed to serve (or continue to serve) on a committee of the institution used in the development of reports if the individual has a conflict of interest that is relevant to the functions to be performed. For more information, see the National Academies’ Web site at www.national-academies.org.

Other considerations. Membership in the three Academies (NAS, NAE, IOM) and previous involvement in National
Academies studies are taken into account in committeeselection. The inclusion of women, minorities, and young professionals are additional considerations.

Specific steps in the committee selection and approval process are as follows:

 Staff solicit an extensive number of suggestions for potential committee members from a wide range of sources, then recommend a slate of nominees.

 Nominees are reviewed and approved at several levels within the National Academies; a provisional slate is then approved by the President of the National Academy of Sciences, who is also the Chair of the National Research
Council.

 The provisional committee list is posted for public comment in the Current Projects System on the Web (http://www4.national-academies.org/cp.nsf).

 The provisional committee members complete background information and conflict of interest disclosure
forms.

 The committee balance and conflict of interest discussion is held at the first committee meeting.

 Any conflicts of interest or issues of committee balance and expertise are investigated; changes to the committee are proposed and finalized.

 Committee is formally approved.

 Committee members continue to be screened for conflict of interest throughout the life of the committee. 

## Ensuring Independent, Objective Advice 

# OUR STUDY PROCESS



# NATIONALACADEMIESTHE



### National Academy of Sciences National Academy of Engineering Institute of Medicine National Research Council

119Project
starts when funding is received Provisional slate approved by NAS President Provisional committee posted for public
comment via Current Projects System Committee balance and expertise evaluated; any conflicts of interest are investigated Final committee formally approved Monitoring for potential conflicts of interest continues 

### STEPS TAKEN TO ENSURE INDEPENDENCE AND OBJECTIVITY

The reports of the National Academies are viewed as beingvaluable and credible because of the institution’s reputation for providing independent, objective, and non-partisanadvice with high standards of scientific and technical quality. Checks and balances are applied at every step in the study process to protect the integrity of the reports and to maintain public confidence in them. The study process can be broken down into four major stages: 1) defining the study; 2) committee selection and approval; 3) committee meetings, information gathering, deliberations, and drafting of the report; and 4) report review.



### STAGE 1. Defining the Study

Before the committee selection process begins, National
Academies’ staff and members of their boards work with sponsors to determine the specific set of questions to be addressed by the study in a formal “statement of task,” as well as the duration and cost of the study. The statement of task defines and bounds the scope of the study, and it serves as the basis for determining the expertise and the balance of perspectives needed on the committee.

The statement of task, work plan, and budget must be approved by the Executive Committee of the National Research Council Governing Board. This review often results in changes to the proposed task and work plan. On occasion, it results in turning down studies that the institution believes are inappropriately framed or not within its purview. 

### STAGE 2. Committee Selection and Approval

Selection of appropriate committee members, individually and collectively, is essential for the success of a study. All committee members serve as individual experts, not as

r epresentatives of organizations or interest groups. Each member is expected to contribute to the project on the basis of his or her own expertise and good judgment. A committee is not finally approved until a thorough balance
and conflict of interest discussion is held at the first meeting, and any issues raised in that discussion or by the public are investigated and addressed.

Careful steps are taken to convene committees that meet the following criteria:

An appropriate range of expertise for the task. The committee must include experts with the specific expertise and experience needed to address the study’s statement of task. One of the strengths of the National Academies is the tradition of bringing together recognized experts fromdiverse disciplines and backgrounds who might not otherwise collaborate. These diverse groups are encouraged to conceive new ways of thinking about a problem.

A balance of perspectives . Having the right expertise is
not sufficient for success. It is also essential to evaluate the overall composition of the committee in terms of different experiences and perspectives. The goal is to ensure that the relevant points of view are, in the National Academies’ judgment, reasonably balanced so that the committee can carry out its charge objectively and credibly. 

## POINT OF VIEW IS DIFFERENT FROM CONFLICT OF INTEREST

A point of view or bias is not necessarily a conflict of interest. Committee members are expected to have points of view, and the National Academies attempt to balance these points of view in a way deemed appropriate for the task. Committee members are asked to consider respectfully the viewpoints of other members, to reflect their own views rather than be a representative of any organization, and to base their scientific findings
and conclusions on the evidence. Each committee member has the right to issue a dissenting opinion to the report if he or she disagrees with the
consensus of the other members.

NRC Governing Board reviews and approves study scope and plan Committee’s first meeting Report is released to the sponsor and the public Committee and National Academies sign off on report

Full committee signs off on draft report Reviewers comment on report

DEFINING THE STUDY COMMITTEE SELECTION AND APPROVAL

COMMITTEE MEETINGS, INFORMATION GATHERING, DELIBERATIONS, AND DRAFTING REPORT REPORT REVIEW 

# → Screened for conflicts of interest. All provisional committee members are screened in writing and in a confidential group discussion about possible conflicts of interest. For this purpose, a “conflict of interest” means any
financial or other interest which conflicts with the service of the individual because it could significantly impair theindividual's objectivity or could create an unfair competitive advantage for any person or organization. The term“conflict of interest” means something more than individual bias. There must be an interest, ordinarily financial, that could be directly affected by the work of the committee. Except for those rare situations in which the National
Academies determine that a conflict of interest is unavoidable and promptly and publicly disclose the conflict of

STAGE 1 STAGE 2 STAGE 4 STAGE 3 

# F or more than 140 years, the National Academies have been advising the nation on issues of science, technology, and medicine. The 1863 Congressional charter signed by President Lincoln authorized this non-governmental institution to honor top scientists with membership and to serve the nation whenever called upon. Today the National Academies— National Academy of Sciences, National Academy of Engineering, Institute of Medicine, and National Research Council

— continue that dual mission.

Like no other organization, the National Academies canenlist the nation’s foremost scientists, engineers, health professionals, and other experts to address the scientific and technical aspects of society’s most pressing problems. Each year, more than 6,000 of these experts are selected to serve on hundreds of study committees that are convened to answer specific sets of questions. All serve without pay.

Federal agencies are the primary financial sponsors of the Academies’ work. Additional studies are funded by state agencies, foundations, other private sponsors, and the National Academies endowment. The Academies provide independent advice; the external sponsors have no control over the conduct of a study once the statement of task and budget are finalized. Study committees gather information from many sources in public meetings but they carry out their deliberations in private in order to avoid political, special interest, and sponsor influence.

Through this careful study process, the National Academies produce 200–300 authoritative reports each year. Recent

r eports cover such topics as the obesity epidemic, the use of forensics in the courtroom, invasive plants, underage drinking, the Hubble Telescope, vaccine safety, the hydrogeneconomy, transportation safety, climate change, and homeland security. Many reports influence policy decisions; some are instrumental in enabling new research programs; others provide program reviews.

120

# GETTING TO KNOW

THE COMMITTEE PROCESS

National Academy of Sciences National Academy of Engineering
Institute of Medicine
National Research Council 

# NATIONALTHEACADEMIES

121

# The National Academy of Sciences was established by Congress more than a
century ago to provide scientific and technological advice to the nation. Over the years, the Academy has evolved to incorporatefour distinguished organizations the National
A cademy of Sciences, the National Academy of Engineering, the Institute Medicine, and the National Research Council. Known collectively as the National Academies, they perform an unparalleled public service by bringing together experts in all areas of scientific and technological endeavor. These experts serve as volunteers to addresscritical national issues and give unbiased advice to the federal government and the public. Most of this advice is provided either by the National Research Council the chief operating arm of the
A cademy of Sciences and the Academy of Engineering or by the Institute of Medicine, which operates under the charter of the National Academy of Sciences and according to ResearchCouncil rules.The National Academies provide science and technology advice in several different forms: written reports reflecting theconsensus reached by an expert study committee; symposia and convocations engaging large audiences in discussion of national issues; proceedings from conferences and workshops; or “white papers” on The National Academy of Sciences was established by Congress more than a century ago to provide scientific and technological adviceto the nation. Over the years, the Academy has evolved to incorporate four distinguished organizations the National Academy of Sciences, the National Academy of Engineering, the Institute of by
USEFUL REFERENCES ABOUT THIS UNIQUE INSTITUTION 2

THE STUDY PROCESS 5

Committee Selection 6

Bias and Conflict of Interest 6

Committee Deliberations 7

Studies of Risk Assessment 8

Institutional Oversight 8

W riting the Report 9

Role of Staff 9

Report Review 10

Confidentiality 10

Public Access 12

DISSEMINATION 14

Congressional Outreach 14

Academies’ News Office 15

National Academies Press 15

Communications Initiative 16

Academies’ Web Site 16

USEFUL REFERENCES 1 7CONTENTS Guidelines for Review: Consensus Reports

The National Academies Study Process: Ensuring Independent, Objective Advice

Roles of the Committee Chair

A Unique National Resource

Revised 2005

Copies of this booklet are available from the Office of News and Public Information

T

el.: 202-334-2138 E-mail: news@nas.edu

Fax: 202-334-2158 Internet: national-academies.org

122

# AC ADEMIESASKED TO SERVE . . .THE NATIONAL 

# Y SO YOU'VE BEEN

ou have been invited to

work on a project at the National

Academies and may be wondering

exactly what your role is as a committee member. This document is a brief

introduction to the institution and is

designed to give you a sense of the

committee process. No two projects

are alike, of course, and people’s

experiences vary. But general policies

and procedures have been developed

to ensure that the time you spend as a

volunteer in service to the nation is

productive and rewarding.

123

# AC ADEMIESTHE NATIONAL 2

The National Academy of Sciences was established by Congress more than a century ago to provide scientific and technological advice to the nation. Over the years, the Academy has evolved to incorporate four distinguished organizations — the National Academy of Sciences, the National Academy of Engineering, the Institute of Medicine, and the National Research Council. Known collectively as the National Academies, they perform an unparalleled public service by bringing together experts in all areas of scientific and technological endeavor. These experts serve as volunteers to address critical national issues and give unbiased advice to the federal government and the public. Most of this advice is provided either by the National Research Council — the chief operating arm of the Academy of Sciences and the Academy of Engineering — or by the Institute of Medicine, which operates under the charter of the National Academy of Sciences and according to Research Council rules. 

## ABOUT THIS UNIQUE INSTITUTIONJD TALASEK

124The National Academies provide science and technology advice in several different forms: written reports reflecting the consensus reached by an expert study committee; symposia andconvocations engaging large audiences in discussion of national issues; proceedings from conferences and workshops; or “white papers” on policy issues of special interest. Each project is conducted or overseen by a committee serving pro bono, whose members are selected for their expertise on the subject.

This booklet focuses primarily on the study committee process, in which a committee of approximately 10 to 20 members with a diverse range of expertise and perspectives is convened to address a particular question or set of questions. Study committees have been assembled, for example, to address various aspects of the AIDS crisis, to consider the use of forensics techniques in the courts, and to offer policy advice on setting priorities for federal funding of research and development. Before a committee meets, one of the volunteer experts is appointed to serve as the committee chair, and a member of the Academies’ staff is assigned as the study director.

Each committee investigates the many facets of the problem described in its statement of task and develops a report of its findings, conclusions, and recommendations, based on the available scientific evidence. This report is subjected to rigorous review by a second groupof independent experts anonymous to the committee. After revisions are made by the committee to satisfy the institution’s review process, the report is transmitted to the agency or organization that sponsored the study, and then released to the public. Names and affiliations of principal reviewers are made public when the report is released.

The National Academies and their operations are distinctive in several ways:

• Committee members serve without compensation, except for reimbursement of expenses.

• Most projects originate from requests made by outside sponsors such as Congress, federal agencies, and foundations. However, volunteer members of the institution’s boards also develop ideas for studies that subsequently are funded by external sources. In addition, the Academies have been building their own endowments through private contributions so that they can address important issues of science and policy without government or foundation support. 3

125

# AC ADEMIESTHE NATIONAL 4 • The institution has no research laboratories. In reaching conclusions and recommendations, study committees hear invited testimony and evaluate published research from the

r elevant scientific literature, as part of an educational process designed to enable the committee to reach consensus. The resulting reports often present a novel synthesis of ideas unique to the committee; these reports are highly regarded by the sponsoringagencies and the public for their thorough analysis and carefully supported recommendations. The reputation of the institution for objectivity, integrity, independence, and competence is one of its most valuable assets. For this reason, procedures designed to ensure excellence apply to each study undertaken.

• The National Academies do not compete in response to federal requests for proposals. Their one-of-a-kind service, not duplicated by other organizations, was reaffirmed in a January 1993 executive order from President Bush.

• The Academies are not subject to the Federal Advisory Committee Act in the way government agencies are; neither are they subject to the Freedom of Information Act. However, the Federal Advisory Committee Act Amendments of 1997 (Public Law 105-153) require the Academies to provide opportunities for public access and involvement in the study process. Those requirements are discussed in the Public Access

section of this booklet.

1265 The core of the institution’s work consists of studies usually of six months to two years in duration and performed under contract to a sponsor or set of sponsors. Each study is conducted by a committee selected expressly for that purpose. The committee meets at intervals to consider its scope of work, to review the relevantscientific evidence, and to develop its findings, conclusions, and recommendations. Once the report has been outlined by the committee, various sections often are written by individual members. The writing process is guided by the committee chair and aided by the committee staff, in particular by the study director.

The charge to the committee — developed before committee members are selected — is the formal statement of the questions to be addressed by the study. This statement defines the study’s scope and issues to be examined. If a committee finds in the course of its work that this description is inadequate, the charge can be formally modified through petition to the Executive Committee of the National Research Council’s Governing Board. Such petitions are carefully examined; a committee that is well-balanced for one purpose may not be appropriately constructed for a modified task. 

## THE STUDY PROCESSJD TALASEK

127

# AC ADEMIES COMMITTEE SELECTION
The search for candidates for committee membership is initiated by staff with input and oversight from the relevant boards. In defining the areas of expertise that should be represented on a committee and identifying individuals qualified to serve, the staff reviews scholarly literature and consults widely with the institution’s members and volunteers, knowledgeable authorities, and professional associations. Sponsors may offer suggestions but do not select

committee members.

Committee members are chosen on the basis of their knowledge and experience in the various aspects of the topic to be investigated, and after careful review are appointed by the chair of the National Research Council, who also is the president of the National Academy of Sciences. The names, affiliations, and short biographies of committee members are posted for public comment in the “Current Projects” area of the institution’s Web site at <national-academies.org>.

BIAS AND CONFLICT OF INTEREST
Committee members serve as individuals, not as representatives of organizations or interest groups. Each person is selected on the basis of his or her expertise and good judgment, and is expected to contribute accordingly to the study.

The credibility of a report can be called into question if the committee that produced it is perceived to be biased. Potential sources of bias and conflict of interest are significant issues thatare taken into consideration in the selection of committee members and are re-examined periodically throughout the study process.

Recognizing that each individual who is knowledgeable about a subject brings his or her own biases and experiences to any study effort, the institution has adopted specific procedures to achieve appropriate balance in the committee membership and to avoid conflicts of interest. At the time of appointment, each committee member is required to list all professional, consulting, and financial connections, as well as to describe pertinent intellectual positions and public statements by filling out a confidential form, “Background Information and Confidential Conflictof Interest Disclosure.” The committee appointment is not finalized until the institution completes a review of information regarding potential conflicts of interest and bias.THE NATIONAL 6

128As part of the process of becoming acquainted with each other and with the task before them, committee members discuss this information in closed session at the beginning of their first meeting and annually thereafter. The information also is reviewed by officials of the institution, and if a potential conflict becomes apparent, the committee member may be asked to resign. In exceptional circumstances, an individual may continue to serve on the committee if theconflict of interest is promptly and publicly disclosed, and the National Academies have determined that the conflict is unavoidable. To fulfill its legal requirement for such public disclosure, the institution posts on its Web site a brief statement describing the unavoidable conflict. When a question of balance arises, the usual procedure is to add members to the committee to achieve the appropriate balance.

COMMITTEE DELIBERATIONS
A successful report is the result of a dynamic group process, requiring that committee members be open to new ideas and innovative solutions, and be willing to learn from one another.

Committees are expected to be evenhanded and to examine all evidence dispassionately. Although all interested parties should be heard and their views given serious and respectful consideration, one of the committee’s primary roles is to separate fact from opinion, analysis from advocacy. Scientific standards are essential in evaluating all arguments and alternatives.

Most committees eventually issue a unanimous report of their conclusions. Members of committees should strive for consensus, but not at the cost of substantially weakening their analyses and conclusions. It may be more valuable in the long run to explain the rationale behind areas of disagreement than to issue unanimous conclusions that are so limited that they fail to contribute to a better understanding of the issue. 7PAUL KENNEDY

129

# AC ADEMIES STUDIES OF RISK ASSESSMENT
Studies involving the assessment of risk are among the most difficult and potentially controversial. Government agencies and others engaged in the management of hazardous conditions and substances often are faced with making policy decisions in the absence of conclusive scientific evidence. Many Research Council and Institute of Medicine studies dealing with risk are requested by federal agencies seeking specific answers on which to base important policy decisions. In studies involving incomplete or inconclusive data, the collective experience and reasoned judgment of committee members become crucial factors in developing conclusions and recommendations.

It is critical for each committee working on a study of risk assessment to distinguish clearly between conclusions based on scientific evidence and those based on informed judgment. In cases in which scientific proof is incomplete, special care is needed to explain how the committee arrived at its conclusions. Moreover, the assumptions used should be explicitly identified and justified. A conscientious effort to be clear in writing the report will help to avoid the potential for misinterpretation when the report is published.

The institution has developed guidelines for committees charged with conductingrisk-related projects (available on the Academies’ intranet site, the AcademyNet). These guidelines emphasize the special care that must be taken in assembling the committee, orienting new committee members, conducting bias discussions, managing the consensus process, handling any minority opinions, and writing the report.

INSTITUTIONAL OVERSIGHT
Every study is subject to oversight, from initial approval to public release of a final report.Whether a study is requested by a government agency or a private organization, or is initiated within the institution itself, the study proposal first must be approved by the Executive Committee of the Research Council’s Governing Board. This group carefully examines the proposal and considers such factors as the importance and timeliness of the question,whether there is an adequate base of scientific knowledge to support the study, the intended audience, the likely impact of the report, and the competence of the institution to

take on the task.

Oversight is provided throughout the duration of a study by the various supervisory entities within the Research Council and the Institute of Medicine, as well as by the committee chair and the staff member serving as study director. Their responsibilities are to ensure that theTHE NATIONAL 8

130committee focuses on its stated tasks, that measures of quality control are enforced, and that the study proceeds on schedule and within budget. Sponsors do not engage in oversight of

the studies.

WRITING THE REPORT
Completing the consensus-building process and writing a report that clearly presents the committee’s findings, conclusions, and recommendations may be the most challenging, yet rewarding aspects of serving on a study committee. The report may well become an important reference for those who formulate public policy. For this reason, the value of a carefully prepared

r eport cannot be overstated.

Although each committee may go about the drafting of its report differently, every report is thecollective product of a group process. A committee member often will draft a chapter or portion of the report, but the “author of record” is the entire committee, and the responsibility for authorship lies with the committee as well. Individual authorship generally is not credited; the

r eport and all copyrights become the property of the National Academy of Sciences.

ROLE OF STAFF
Each committee is assisted in its work by highly qualified staff members who facilitate the work ofthe committee during the conduct of the study. When committee and staff form a close professional partnership, the experi ence can be exhilarating for everyone involved.

Staff help to create the objective atmosphere in which the committee’s deliberations take place. In addition, staff are responsible for ensuring that institutional procedures and practices are followed throughout the study, and that the study stays on schedule and within budget.

Staff members assist with many aspects of assembling the report, including researching, writing, integrating portions written by others, and ensuring consistent style and format. However,

the conclusions and recommendations are those of the committee. Staff do not insert their personal conclusions or recommendations into the report. 9

131

# AC ADEMIESTHE NATIONAL 10 REPORT REVIEW
Like all good science, reports should be based on fact and rigorous analysis. All of the institution’s reports — whether products of studies, summaries of workshop proceedings, or abbreviated documents — must undergo an independent review by anonymous experts who were not involved in the report’s preparation.

Report review is an integral and constructive part of the study process. It is the final opportunity for committees to test their reasoning, conclusions, and recommendations before release of the

r eport to the public.

The Report Review Committee (RRC) ensures that an independent review has been
conducted, and that:

• the report addresses the approved study charge and does not go beyond it; • the findings are supported by the evidence and arguments presented; • the exposition and organization are effective; and • the tone of the report is impartial, and sensitive policy issues are treated with appropriate care.

The report may not be transmitted to the sponsor or released to the public until review has been completed to the satisfaction of the Report Review Committee. Details of the quality standards followed by reviewers can be found in the RRC document Guidelines for Review: Consensus Reports . Once the report is released, names and affiliations of principal reviewers are made public.

CONFIDENTIALITY
During more than a century of service, the institution has earned a reputation for providing independent, expert advice. Procedures and practices have evolved that protect committees from outside pressures and thereby safeguard the credibility and integrity of their work.

Committee meetings, particularly as the committee gathers information, are usually open to interested individuals and the news media. However, meetings are closed when the committee is deliberating to develop its findings and during discussion of financial and personnel matters. Closed meetings are not open to the public or to any person who is not a committee member or an official, agent, or employee of the Academies.

132Reports are the product of the institution, not of the committee alone. Committee deliberations, drafts of the report in progress, and tentative conclusions all are confidential until a completed

r eport passes through review and receives sign-off by the Report Review Committee and by the major unit responsible for the study. Committee members are expected to reject any

r equests for early briefings or media interviews on the committee’s findings, and to treat committee deliberations and draft products as confidential.

A cardinal rule to keep in mind: Until the review process has been satisfactorily completed,the document is not an official report of the National Academies. Conclusions and recommendations can change up to the final sign-off; premature briefings for sponsors or others outside the institution may lock committees into a position not fully supported by the evidence. Early briefings also damage the final report by subjecting the committee to the accusation that it permitted the sponsor to preview and approve the conclusions and recommendations — a serious charge that undermines the independence and integrity of both the committee and the institution. In such cases, the hard work of the committee can be discredited, diminishing the report’s value to the sponsor and to the nation.

Until the report is publicly released, committee members should limit public comment to the following:

• the scope of the project and what the committee has been asked to do; • the name of the sponsor and estimated cost of the study, if known; and • the makeup of the committee, including names and affiliations of

committee members. 11GREG HADLEY

133

# AC ADEMIES This and other relevant information can be found on the institution’s Web site at <national-academies.org> and in its public access files. These files, which include items such as materials presented to the committee in data-gathering meetings open to the public, are available for public review.

PUBLIC ACCESS
Legislation passed in late 1997 protects the Academies from government control under the Federal Advisory Committee Act. But in doing so, it explicitly requires the Academies to ensure public

access to committee activities.

For example, any meeting of a committee at which anyone other than committee members or officials, agents, or employees of the institution is present — whether in person, by telephone, or teleconference— is considered a “data-gathering committee meeting.” Except as determined and approved in advance by the Academies’ leadership, all data-gatheringcommittee meetings are open to the public. Datagathering meetings that involve committee consideration of classified, proprietary, or personal-privacy information, however, exemplify situations in which meetings are not open to the public.

To facilitate the process of informing the public abouta committee’s work and enabling interested individuals to attend open data-gathering sessions, anadvance announcement must be posted — preferably 14 days before the meeting — on the institution’s Web site.

Data-gathering committee meetings should be

r egarded as on the record. Therefore, whether or notTHE NATIONAL 12

134r epresentatives of the news media are in attendance, the chair of the meeting should advise everyone present of the nature and purpose of the meeting. Statements of this type are necessary to help ensure that participants and observers do notmisinterpret the purpose of the meeting, or prematurely interpret the discussion to be the positions ofindividual participants, the committee, or the institution. For guidance on assembling introductory

r emarks for the chair, see “Setting the Ground Rules at an Open Meeting to Which the Public and Press
Have Been Invited as Observers,” available on the AcademyNet.

To acquaint the public with the background of committee members, the chair should ask each member to state briefly, in open session at the first data-gathering committee meeting, those aspects of his or her background, experience, expertise, and previously stated positions that appear relevant to the functions to be performed by the committee. Committees also should create opportunities that facilitate the gathering of as wide a range of views as possible, such as having a session for public comments at a data-gathering committee meeting or soliciting comments in writing or via e-mail from interested members of the public.

W ithin 10 days following a closed committee meeting, the Academies will post a brief summary of the meeting on the Web, listing the committee members present, the topics discussed, and materials made available to the committee. This summary will not
disclose the substantive content, conclusions,

r ecommendations, discussion of draft reports, or
any report r eview comments. 13JD TALASEK

135

# AC ADEMIESTHE NATIONAL 14

Once a report has successfully completed review and been thoroughly edited, Academies staff working with the Office of Congressional and Government Affairs and the Office of News and Public Information develop a release timeline. At this stage it is critical to observe carefully the institution’s rules of confidentiality until the report is ready for release to the public.

CONGRESSIONAL OUTREACH
The Office of Congressional and Government Affairs (OCGA) is responsible for dissemination and outreach to the congressional and executive branches of government. One component of this is congressional and sponsor briefings. These briefings take place up to 24 hoursbefore the public release of a report. Dissemination of reports that are congressionally mandated involves specific guidelines for briefing members of Congress. For other reports, OCGA informs concerned members of Congress and the appropriate congressional committees.

OCGA also monitors congressional activities on issues that affect the National Academies and helps committee members or staff prepare and submit testimony for Congress. 

## DISSEMINATION

136AC ADEMIES’ NEWS OFFICE
The Office of News and Public Information (ONPI) is the liaison between the Academies and the news media and general public. Many reports of the National Academies are newsworthy, and the media serve as an important channel for disseminating the content of reports.

For each report, ONPI works closely with study staff to develop a release plan that may include a news release; public briefing; embargo arrangement; outreach to targeted media; and mediapreparation for committee members to help them speak comfortably with journalists and handle potentially difficult inquiries.

Committee members or staff who receive a call from a reporter concerning a pending report generally should feel free to respond, but in a limited way. Journalists may be provided with a project’s statement of task, the committee roster, the identity of the sponsor(s), and the cost of the study, as well as general information about the background and scope of the project. Findings and recommendations, however, must be kept confidential until the report is released.

ONPI should be informed of substantive conversations with the news media, especially if there is any problem. Staff or committee members seeking guidance about media relations may consult with ONPI at any time. If a reporter is asking leading questions or inquiring about controversial issues, it often is best to delay answering until after conferring with ONPI.

ONPI also publicizes Academies reports through weekly and monthly electronic newsletters and the institution’s magazine, The National Academies In Focus

.

NATIONAL ACADEMIES PRESS
The National Academies Press (NAP) is the publisher for the institution. NAP offers a wide range of services,including publication planning, editing, printing, marketing, and distribution. It publishes nearly 200 committee

r eports each year, ranging from pre-publication photocopies to full-color, high-quality trade books that are

marketed and sold around the world. The NAP Web site at <www.nap.edu> makes all of these publications
available online — more than 3,000 books and 500,000 book pages in 2005. Most books published by NAP are also offered as PDF files, many of which are free to the public. 15

137Current committee members are permitted free access to all available PDF files and should contact study staff to make the necessary arrangements; they also receive a 25 percent discount on all books purchased from NAP for personal use.

COMMUNICATIONS INITIATIVE
The National Academies strive to disseminate the results of their work to a wide range of audiences. The Office of Communications was formed in 2001 to help the institution accomplish this. Specifically, this office works in partnership with Academies program units to engage the public with the institution’s work, create new products and services for different audiences, build stronger relationships with existing and potential sponsors, and harness the Web as an effective communication vehicle. Committees are encouraged to identify potential audiences for their studies early in the process, discuss how to reach those audiences, and consider ways of expanding public interest throughout the study process.

AC ADEMIES’ WEB SITE
Information about the institution’s work is available on the Web at <national-academies.org>. A broad range of material can be found online, including current project information, news

re leases, and full-text reports.



# AC ADEMIESTHE NATIONAL 16

138

# The National Academy of Sciences was established by Congress more than a
century ago to provide scientific and technological advice to the nation. Over the years, the Academy has evolved to incorporatefour distinguished organizations the National
A cademy of Sciences, the National Academy of Engineering, the Institute Medicine, and the National Research Council. Known collectively as the National Academies, they perform an unparalleled public service by bringing together experts in all areas of scientific and technological endeavor. These experts serve as volunteers to addresscritical national issues and give unbiased advice to the federal government and the public. Most of this advice is provided either by the National Research Council the chief operating arm of the
A cademy of Sciences and the Academy of Engineering or by the Institute of Medicine, which operates under the charter of the National Academy of Sciences and according to ResearchCouncil rules.The National Academies provide science and technology advice in several different forms: written reports reflecting theconsensus reached by an expert study committee; symposia and convocations engaging large audiences in discussion of national issues; proceedings from conferences and workshops; or “white papers” on The National Academy of Sciences was established by Congress more than a century ago to provide scientific and technological adviceto the nation. Over the years, the Academy has evolved to incorporate four distinguished organizations the National Academy of Sciences, the National Academy of Engineering, the Institute of by
USEFUL REFERENCES ABOUT THIS UNIQUE INSTITUTION 2

THE STUDY PROCESS 5

Committee Selection 6

Bias and Conflict of Interest 6

Committee Deliberations 7

Studies of Risk Assessment 8

Institutional Oversight 8

W riting the Report 9

Role of Staff 9

Report Review 10

Confidentiality 10

Public Access 12

DISSEMINATION 14

Congressional Outreach 14

Academies’ News Office 15

National Academies Press 15

Communications Initiative 16

Academies’ Web Site 16

USEFUL REFERENCES 1 7CONTENTS Guidelines for Review: Consensus Reports

The National Academies Study Process: Ensuring Independent, Objective Advice

Roles of the Committee Chair

A Unique National Resource

Revised 2005

Copies of this booklet are available from the Office of News and Public Information

T

el.: 202-334-2138 E-mail: news@nas.edu

Fax: 202-334-2158 Internet: national-academies.org

139

# GETTING TO KNOW

THE COMMITTEE PROCESS

National Academy of Sciences National Academy of Engineering
Institute of Medicine
National Research Council 

# NATIONALTHEACADEMIES

140

# CONSENSUS REPORTS Repo R t Review Committee 

# GUIDELINES FOR REVIEW 

# NATIONALTHEACADEMIES may not be released to the sponsors or the public, normay its findings be disclosed, until after the review process has been satisfactorily completed and all authors have approved the revised draft.



### C ON f IDENTIALIT y AND A NON y MIT y To encourage reviewers to express their views freely, the review comments are treated as confidential documents and are given to the authors of the report with identifiers removed. After submitting their comments, reviewers are asked to return or destroy the draft manuscript and to refrain from disclosing their comments or the contents of the draft. The names and affiliations of participants inthe review will be made public when the report is released (usually by acknowledgment in the printed report), but their comments remain confidential. Even after release of the report, reviewers should not divulge their comments or any changes made to the draft manuscript.These restrictions are imperative in safeguarding the integrity of the institutional review process.



### S UPPORTIN g Ev IDENCEThe rationale for any findings, conclusions, and recommendations should be fully explained in the report. This explanation might include references to the literature, analysis of data, or a description of the pros and consof the range of alternatives and the reasons for preferring a particular option. Failure to document conclusions and recommendations adequately is the most common shortcoming of draft reports. Of particular concern are recommendations calling for organizational changes or budgetary increases within government agencies or for additional work for the National Academies. In general,such recommendations should be avoided unless specifically called for in the study charge.



### S UMMARIES AND Ex ECUTI v E S UMMARIESEvery study report must contain a Summary or an Executive Summary that clearly and concisely communicates the main message of the report to its intended audiences, including non-experts and readers who do not read the rest of the report. The Summary or Executive Summary may not distort or go beyond the content of the report. It must clearlyidentify the study charge, but need not include all conclusions and recommendations. An Executive Summary should typically be less than 800 words and not exceed 1,000 words. It may appear alone or be followed by a Summary. A Summary should typically be less than 3,000 words and notexceed 5,000 words unless preceded by an Executive Summary (which may be called an “abstract” or “synopsis” when it precedes a Summary).



### C ONSENSUS AND D ISSENT
National Academies committees strive for consensus, but on rare occasion--despite extensive deliberations--one or more committee members may not concur with the views of the majority. Matters of disagreement should be addressed forthrightly in the report. As a final recourse, a committee member may choose to prepare a brief dissent (no morethan 5,000 words) succinctly describing the issues of contention and the arguments in support of the minority view. This statement should be included as an appendix to the draft report, with reference to it in the introductory text and Table of Contents. A dissent may not address issues outside the study charge, misrepresent the majority’s views, or containother inaccuracies. Any questions regarding the appropriateness of material included in a dissent shall be referred tothe RRC chair. Although reviewers’ comments on the statement are given to its author for consideration, no formal written response is required. 

### 4 4 3.85

141

### R E v IE w C RITERIA National Academies reports cover a broad range of topics and appear in a variety of different forms. Although no rigid set of criteria is likely to be applicable to all reports,reviewers may find the following questions useful in formulating their comments. (A separate set of criteria is used for “letter reports” and other abbreviated documents.)

1. Is the charge clearly described in the report? Are all aspects of the charge fully ad- dressed? Do the authors go beyond their charge or their expertise?

2. Are the conclusions and recommendations adequately supported by evidence,analysis, and argument? Are uncertainties or incompleteness in the evidence explicitly recognized? If any recommendations are based on value judgments or the collective opinions of the authors, is this acknowledged and are adequate reasons given for reaching those judgments? If the report is based on a sworkshop, are findings and conclus ions attributed to either an individual or an NRC committee? 3. Are the data and analyses handled competently? Are statistical methods applied appropriately?

4. Are sensitive policy issues treated with proper care? For example, if the report contains recommendations pertaining to the reorganization of an agency or the creation of a new institutional entity, are the advantages and disadvantages of alternative options, including the status quo, considered? consistent with other sections of the report?

5. Are the report’s exposition and organization effective? Is the title appropriate?

6. Is the report fair? Is its tone impartial and devoid of special pleading?

7. Does the executive summary concisely and accurately describe the key findings and recommendations? Is it consistent with other sections of the report?

8. Are signed papers or appendices, if any, relevant to the charge? If the report relies on signed papers to support consensus findings or recommendations, do the papers meet criterion 3 above?

9. What other significant improvements, if any, might be made in the report?

In providing comments, reviewers are encouraged to distinguish issues they consider to be of general/major concern from other, less significant points

. 

### P URPOSE The purpose of review is to assist the authors in making their report as accurate and effective as possible and toensure that they and the National Academies are creditably represented by the report published in both their names. Review not only fulfills the institutional obligation toexercise oversight, but also provides the authors with preliminary reactions from a diverse group of experts and, as a result, enhances the clarity, cogency, and credibility of the final document. Reviewers are asked to consider whether in their judgment the evidence and arguments presented are sound and the report is fully responsive tothe study charge, not whether they concur with the findings.



### P RO CESS The report review process is overseen by the Report Review Committee (RRC), made up of approximately30 members of the National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. The process is managed by the commission, board, or office responsible for institutional oversight of the project. This administrative unit, in consultation with the RRC, appoints a group of independent reviewers with diverse perspectives on key issues considered in the report. A draft report is sent to reviewers only after all authors have indicated that they are satisfied with itsform and content. Reviewers receive the complete report (including front matter, preface, executive summary, and all appendices), along with the statement of task and this brochure. Reviewers are asked to provide written comments on any and all aspects of the draftreport, but to pay particular attention to the review criteria set forth in the final section of this brochure. The authors are expected to consider all review comments and to provide written responses, which are evaluated bythe monitor (appointed by the RRC) and/or review coordinator (appointed by the administrative unit). A report 

## GU i D e L i N e S F o R t H e R eview o F 

## NATIONAL ACADEMIES REPORTS

Any National Academies report (including meeting summaries, signed papers, letter reports, or other study products) must be reviewed by a diverse group of experts other than its authors before it may be released outside the institution.This independent, rigorous review is a hallmark that distinguishes the National Academies from many other organizations offering scientific and technical advice on issues of national importance.NATIONAL THE



### ACADEMIES 4 4 3.85

142